Summary for Python Codes

Compilation of notes for coding with python

Learning points

• Linking with google sheets to have a searchable dictionary for coding with python

• Unfortunately, new line within a cell does not show up in DT below, so the formatting isn’t great.

• Learning how to use Python in R, using reticulate package

CODES

Terminal

Base Python

Show 102550100 entries

Search:

	Purpose	Code
1	Help and documentation	help(len)
2	Help and documentation	?round
3	Installation	conda install package_name
4	Installation	pip install package_name
5	Installation	conda update package_name
6	Installation	pip install --upgrade package_name
7	magic command for plots to appear in notebook	%matplotlib
8	append an element to the list	a = [1,2,3] a.append(4). # append 4 a --> [1,2,3,4]
9	define function to append an element to the list	# create function def append_element(somelist, element): somelist.append(element) # run function data = [1,2,3] append_element(data, 4) data --> [1,2,3,4]
10	check type of variable	a = 5 type(a) --> int

Showing 1 to 10 of 177 entries

Previous12345…18Next

Other packages

Workflow for normal EDA

• Make a copy of the original dataset before making any changes.

• Get a feel of data. Use glimpse() for R, Use my_df.head().T for python. Other commands include: df.tail(), df.index(), df.columns, df.describe(), df.info, df.shape, df.sample(5, random_state = 0)

• Check datatype to see if there is any need to change to the right dtype

• Check which columns are redundant, and drop if not necessary df_dropped_age_sex = df.drop([‘Age’, ‘Sex’], axis = 1).head(1).T

• Check for duplicated rows (only those that match perfectly across all columns) data_iris_preprocessed = data_iris.drop_duplicates()

• Check descriptive statistics, and get summary statistics to see if there is a need for scaling, outliers.

• Check for unique values to see if any factor-recoding should be done.

• Boxplot/Histogram to see if there are any outliers

• Check class distribution

• Check correlation

• Check skews of univariate plots. Histograms may be used to show distribution for numerical data.

• Check for outliers - ascertain whether it is due to measurement error, data corruption, or it is a true outlier (requires domain knowledge). It is not a good idea to remove outliers without knowing why. One way to remove outliers is to use the IQR method to check if they are outliers.

• For categorical data, counts (barplot) may be used.

• For bivariate relationships (numerial/numerical): use scatterplot to show relationship, or jointplot to show both scatterplot and histogram.

• For bivariate relationships (numerical Y, categorical X): use boxplots

• For multivariate relationships, use pairplot

Show 102550100 entries

Search:

	No.	Package	Purpose	Code
1	1	numpy	create data	data = np.random.randn(2, 3) # create 2 rows, 3 cols random numbers
2	2	numpy	dimensions of data	data.shape --> (2, 3)
3	3	numpy	data type	data.dtype --> dtype('float64')
4	4	numpy	create array from nested lists	arr = np.array(your_nested_list)
5	5	numpy	basic indexing for one-dimensional array	arr[5]
6	6	numpy	transpose	my_array.T
7	7	numpy	compute absolute value element wise	np.abs(my_array)
8	8	numpy	compute square root	np.sqrt(my_array)
9	9	numpy	compute exponent (e^x)	np.exp(my_array)
10	10	numpy	compute natural log (base e), log base 10, log base 2 and log (1 + x)	np.log(my_array) np.log10(my_array) np.log2(my_array) np.log1p(my_array)

Showing 1 to 10 of 279 entries

Previous12345…28Next

LIBRARIES

Show 102550100 entries

Search:

	library	code	purpose
1	numpy	import numpy as np	numerical computing
2	pandas	import pandas as pd	work with structured or tabular data
3	matplotlib	import matplotlib as plt	creating plots
4	scipy	import seaborn as sns	mathematical computing
5	scikit-learn		machine learning toolkit
6	statsmodels	import statsmodels as sm	regression analysis models eg linear models, ANOVA, time series, non-parametric methods, visualization of stats model results
7	datetime	from datetime import datetime, date, time	built in library, for working with dates and times
8	pprint	from pprint import pprint	pretty print
9	matplotlib.pyplot	import matplotlib.pyplot as plt	basic plotting module in python. https://matplotlib.org/index.html
10	warnings	import warning	warnings

Showing 1 to 10 of 44 entries

Previous12345Next

RETICULATE

Installing libraries

library(reticulate)

## check configuration
# py_config()
# y_discover_config()

# install packages - https://rstudio.github.io/reticulate/articles/python_packages.html
# py_install('pandas')
# py_install('scipy')
# py_install('matplotlib.pyplot')
# py_install('seaborn')
# py_install('scikit-learn')

Creating variables in python

Import libraries Indicate chunk code with Python instead of R.

import numpy as np
my_python_array = np.array([2,4,6,8])
my_python_array

array([2, 4, 6, 8])

Loading python variables into R

my_r_array <- py$my_python_array

Loading R variables back into python

my_python_array_2 = r.my_r_array

References

• Python Data Science Handbook, Jake VanderPlas
• Python for Data Analysis, Wes McKinney
• Heicoders AI100 Course
• https://rstudio.github.io/reticulate/
• https://anderfernandez.com/en/blog/how-to-use-python-in-r/
• https://raw.githubusercontent.com/rstudio/cheatsheets/main/reticulate.pdf