how to shift a pandas multiindex series?

Use groupby/shift to apply the shift to each group individually: (Thanks to Jeff for pointing out this simplification.)

In[60]: df['beyer_shifted'] = df.groupby(level = 0)['beyer'].shift(1);
df
Out[61]:
   line_date line_race beyer beyer_shifted
Last Gunfighter 2013 - 09 - 28 10 99 NaN
Last Gunfighter 2013 - 08 - 18 10 102 99
Last Gunfighter 2013 - 07 - 06 8 103 102
Paynter 2013 - 09 - 28 10 103 NaN
Paynter 2013 - 08 - 31 10 88 103
Paynter 2013 - 07 - 27 8 100 88

Suggestion : 2

How it works: df.unstack() moves the values in the nested 0, 1, 2, 3, 4 to the homonym columns, and df.stack() recovers the original nested index., 6 days ago pandas.Series.shift¶ Series. shift (periods = 1, freq = None, axis = 0, fill_value = None) [source] ¶ Shift index by desired number of periods with an optional time freq.. When freq is not passed, shift the index without realigning the data. If freq is passed (in this case, the index must be date or datetime, or it will raise a NotImplementedError), the index will be increased using the ... , 1 week ago Aug 31, 2021 · Pandas dataframe.shift () function Shift index by desired number of periods with an optional time freq. This function takes a scalar parameter called the period, which represents the number of shifts to be made over the desired axis. This function is very helpful when dealing with time-series data. freq : DateOffset, timedelta, or time rule ... , 1 week ago Shift index by desired number of time frequency increments. This method is for shifting the values of datetime-like indexes by a specified time increment a given number of times. Number of periods (or increments) to shift by, can be positive or negative. Frequency increment to shift by. If None, the index is shifted by its own freq attribute.

start = datetime(2012, 1, 1) end = datetime(2012, 4, 1) rng = pd.date_range(start, end) ts = pd.Series(np.random.randn(len(rng)), index = rng)

ts_mi.unstack().shift(2, freq = 'D').stack()

start = datetime(2012, 1, 1) end = datetime(2012, 4, 1) rng = pd.date_range(start, end) ts = pd.Series(np.random.randn(len(rng)), index = rng)

ts.shift(2, freq = "D")

mi = [(dt, i) for dt in rng
   for i in range(5)
] ts_mi = pd.Series(np.random.randn(len(mi)), index = pd.MultiIndex.from_tuples(mi))

2012 - 01 - 01 0 - 0.805353 1 1.467167 2 - 1.207204 3 1.658394 4 1.497559 2012 - 01 - 02 0 - 0.742510 1 0.764594 2 0.558660 3 - 0.479370 4 0.653849...

Suggestion : 3

If you want to shift your column or subtract the column value with the previous row value from the DataFrame, you can do it by using the shift() function. It consists of a scalar parameter called period, which is responsible for showing the number of shifts to be made over the desired axis. It is also capable of dealing with time-series data.,Example2: The example shows how to fill the missing values in the DataFrame using the fill_value.,axis: 0 is used for shifting the index, whereas 1 is used for shifting the column.,JavaTpoint offers college campus training on Core Java, Advance Java, .Net, Android, Hadoop, PHP, Web Technology and Python. Please mail your requirement at [email protected] Duration: 1 week to 2 week

   a_data b_data c_data
   0 NaN NaN NaN
   1 NaN NaN NaN
   2 45.0 26.0 22.0
   3 28.0 37.0 19.0
   4 39.0 41.0 11.0

   a_data b_data c_data
   0 70 70 45
   1 70 70 28
   2 70 70 39
   3 70 70 32
   4 70 70 18

Suggestion : 4

October 31, 2021March 8, 2022

Let’s take a look at how the Pandas shift method works and what its arguments are:

df.shift(
   periods = , # Number of periods to shift freq = , # Shift based on timeseries data fill_value = , # What to fill missing data with after shifting axis = # To shift rows or columns
)

To follow along word for word with this tutorial, I have provided a sample Pandas Dataframe that you can load. Simply copy and paste that code below into your favourite code editor and we can get started:

# Load a Sample Pandas Dataframe
import pandas as pd

df = pd.DataFrame.from_dict({
   'Name': ['Nik', 'Jane', 'Kate', 'Evan', 'Max', 'Kevin', 'Luke'],
   'Amount': [100, 200, 210, 120, 70, 95, 90]
})

print(df.head())

# Returns:
   # Name Amount
# 0 Nik 100
# 1 Jane 200
# 2 Kate 210
# 3 Evan 120
# 4 Max 70

Let’s try moving out records down by one row using Pandas:

# Shift an Entire Pandas dataframe
import pandas as pd
df = pd.DataFrame.from_dict({
   'Name': ['Nik', 'Jane', 'Kate', 'Evan', 'Max', 'Kevin', 'Luke'],
   'Amount': [100, 200, 210, 120, 70, 95, 90]
})

# Print the Original Dataframe
print('Original dataframe')
print(df.head())

# Shift the dataframe and reprint it
df = df.shift(periods = 1)
print('\nShifted Dataframe')
print(df.head())
# Returns:
   # Original dataframe
# Name Amount
# 0 Nik 100
# 1 Jane 200
# 2 Kate 210
# 3 Evan 120
# 4 Max 70

# Shifted Dataframe
# Name Amount
# 0 NaN NaN
# 1 Nik 100.0
# 2 Jane 200.0
# 3 Kate 210.0
# 4 Evan 120.0

In the example below, we’ll shift the Amount column down one record and call it Amount (Shifted):

# Shift a single column in a Pandas dataframe
import pandas as pd
df = pd.DataFrame.from_dict({
   'Name': ['Nik', 'Jane', 'Kate', 'Evan', 'Max', 'Kevin', 'Luke'],
   'Amount': [100, 200, 210, 120, 70, 95, 90]
})

df['Amount (Shifted)'] = df['Amount'].shift(periods = 1)

print(df.head())
# Returns:
   # Name Amount Amount(Shifted)
# 0 Nik 100 NaN
# 1 Jane 200 100.0
# 2 Kate 210 200.0
# 3 Evan 120 210.0
# 4 Max 70 120.0

Let’s see how we can again shift the Amount data down a row and fill the value with 100:

# Shift a single column in a Pandas dataframe and fill missing data
import pandas as pd
df = pd.DataFrame.from_dict({
   'Name': ['Nik', 'Jane', 'Kate', 'Evan', 'Max', 'Kevin', 'Luke'],
   'Amount': [100, 200, 210, 120, 70, 95, 90]
})

df['Amount (Shifted)'] = df['Amount'].shift(periods = 1, fill_value = 100)

print(df.head())
# Returns:
   # Name Amount Amount(Shifted)
# 0 Nik 100 100
# 1 Jane 200 100
# 2 Kate 210 200
# 3 Evan 120 210
# 4 Max 70 120