Pandas

Question 1

Filling NA’s in one column with another

fullDf[‘forecast_date’] has NA fill with fullDf.day

Answer 1

fullDf[‘forecast_date’].fillna(fullDf.day, inplace=True)

Question 2

calls_final.calls_tw has NaN values. replace them with 5

Answer 2

calls_final.calls_tw=calls_final.calls_tw.map(lambda x: x if np.isfinite(x) else 0)

Question 3

Add a series of integers to df_test1[‘forecast_date’]

Answer 3

df_test1['add']=pd.to_timedelta(df_test1['add'], unit='D')
# convert integers to days this way
df_test1['forecast_date']=df_test1['forweek']+df_test1['add']

Question 4

Calculate day of week from a date python

df[‘date’] is a series of dates

Answer 4

df['date'].dt.weekday_name
#df['date'].dt.dayofweek will give numbers Monday :0

Question 5

Sort dataframe ‘df’ by index and save

Answer 5

df.sort_index(inplace=True)

Question 6

Get today’s date and

Convert into string datetime object into ‘2017-07-17’ format

Answer 6

Today = datetime.now()
Today = Today.strftime("%Y-%m-%d")

Question 7

summary for a dataframe df

Answer 7

train. info() #will give a summary for the entire data

train. describe() ##will give a summary for continuous variables in the data

Question 8

filter df for having only values 3 and 6,9 in column A

Answer 8

df[df[‘A’].isin([3, 6])]

Question 9

drop rows which have indices 1 and 3 in df

Answer 9

df.drop(df.index[[1,3]])

Question 10

Convert series df[‘date’] to datetime objects df

Answer 10

df[‘date’]=df[‘date’].map(lambda x: pd.to_datetime(x,dayfirst=True))
df.date=pd.to_datetime(df.date,dayfirst=True)

Question 11

Reset index of dataframe df

Answer 11

df = df.reset_index()

del df[‘index’]

Question 12

Check if col1 and col2 in df are equal

Answer 12

df[‘col1’].equals(df[‘col2’])

Question 13

Convert series df.col1 to a list

Answer 13

list1=df.col1.tolist()

Question 14

Dropping rows by null values in a column df.col1

Answer 14

df = df[np.isfinite(df.col1)]
df=df[pd.notnull(df.col1)]

df=df[pd.isnull(df.col1)] ##keeping only those rows

Question 15

Filtering a dataframe df by column Gender

Answer 15

df[df[Gender]==’Male’]

Question 16

Filtering a dataframe df by two columns Gender and Year

Answer 16

df[(df[Gender]=='Male') & (df[Year]==2014)]
#Dont forget th round brackets

Question 17

Delete col1 from datafrmae df

Answer 17

del df[‘col1’]

df.drop(df.columns[[0, 1, 3]], axis=1) # delete columns by number

Question 18

Check datatype of col1 or of whole dataframe

Answer 18

df. dtypes

df. col1.dtype

Question 19

sort dataframe df according to col1 values

then according to col1 and col2 values

Answer 19

#Sorting the dataframe based on values of one column 
df.sort_values(by='col1',ascending=True)

#based on two columns
df.sort_values(['col1', 'col2'], ascending=[True, False])

Question 20

Print variable and string together

String : hello Variable: Name

Answer 20

print “I have %s” % Name

Question 21

correlation between df.col1 and df.col2

Answer 21

np.corrcoef(df.col1,df.col2)[0,1]

Question 22

Rename col1 of df to col2

Answer 22

df=df.rename(columns = {‘col1’:’col2’})

Question 23

groupby by categorical col3 and aggregate mean

Answer 23

gb =df.groupby(df.col3)
gb.agg(‘mean’)

gb.agg({‘col1’: ‘sum’,’col2’:’mean’})

Question 24

mapping df.country to create df.capital

Answer 24

map1={‘India’:’Delhi’,’Canada’:’Ottawa’}

df[‘capital’] = df[‘country’].map(map1)

Question 25

drop duplicate rows from a df

Answer 25

df.drop_duplicates() #just drops duplicate rows #just keep the first one

Question 26

drop duplicates from a df for a column col1

Answer 26

df.drop_duplicates(['col1']) #drops duplicates by a single column #just keep the first one

Question 27

Drop duplicates from a df for a column col1. | Keeping the last one

Answer 27

df.drop_duplicates(['col1'],keep='last') #take the last value of duplicate

Question 28

pivot dataframe

Answer 28

df_piv = df.pivot(index='date',columns='variable',values='value')

Question 29

all type of merges?

Answer 29

a1=pd.merge(dframe1,dframe2) #default merge inner join on some column a=pd.merge(dframe1,dframe2,on='key') #inner join b=pd.merge(dframe1,dframe2,on='key',how='left') #left join c=pd.merge(dframe1,dframe2,on='key',how='outer') #outer join d=pd.merge(df_left, df_right, on=['key1', 'key2'], how='outer') #on multiple keys e=pd.merge(left, right, left_on='key1', right_on='key2')

Question 30

read csv and write csv syntax?

Answer 30

``` a=pd.read_csv('lec25.csv') b=pd.read_table('lec25.csv',sep=',') c=pd.read_csv('lec25.csv',header=None) d=pd.read_csv('lec25.csv',header=None,nrows=2) dframe1.to_csv('mytextdata_out.csv') ```

Question 31

concatenate df1 and df2

Answer 31

pd.concat([df1,df2])

Question 32

create a dataframe df

Answer 32

from numpy.random import randn df1=DataFrame(randn(25).reshape((5,5)),columns=list('abcde'),index=list('12345')) dframe2 = DataFrame({'key':['Q','Y','Z'],'data_set_2':[1,2,3]})

Question 33

Pivot df syntax?

Answer 33

long to wide is pivot. Pivot takes 3 arguments with the following names: index, columns(cat) , and values(num) entries inside the column(cat) will be used to create new columns index will have distinct values values will go inside the table p = d.pivot(index='Item', columns='CType') If you omit values all numerical columns in the datframe will be used. multi index will be created

Question 34

Unpivot

Answer 34

wide to long is unpivot/melt df = pd.DataFrame({'A': {0: 'a', 1: 'b', 2: 'c'}, 'B': {0: 1, 1: 3, 2: 5}, 'C': {0: 2, 1: 4, 2: 6}}) pd.melt(df, id_vars=['A'], value_vars=['B','C'], var_name="Person", value_name="Score") ##set the variables (columns) that we want to leave unaffected as id_vars. All variables not included in this list will become rows in a new column (which has the name given by “var_name”) if you do not specify value_vars.