pandas, NumPy, and SciPy for Biology Data
This page teaches the Python workflow directly: read a dataset, clean it, summarize groups, run statistical tests, and interpret the result for an EBS poster.
The Python Mental Model
Python analysis is a conversation between a table and a question. Your job is to keep each step visible: load the data, inspect it, clean it, summarize it, test it, and write down what the result means.
Three Libraries, Three Jobs
- pandas handles tables. A pandas table is called a DataFrame.
- NumPy handles numeric arrays and calculations.
- SciPy handles statistical tests and scientific computations.
Teacher note: Do not start with a statistical test. Start by asking, "What is one row? What does each column mean? Which rows should be compared?"
Useful Python Libraries and Commands
You do not need every Python library. For most EBS dry-lab projects, start with pandas for tables, NumPy for numeric calculations, and SciPy for statistics. Add other libraries only when the research question genuinely needs them.
Which Libraries Are Useful?
| Library | Use it for | When it matters in biology research |
|---|---|---|
| pandas | Tables, CSV files, cleaning, filtering, grouping, merging | Any project with sample metadata, public-health tables, experiment results, or spreadsheet-like data |
| NumPy | Arrays, numeric calculations, means, standard deviations, percentiles | When columns need mathematical operations or you need clean numeric inputs for statistics |
| SciPy | Statistical tests, correlations, probability distributions, scientific calculations | When you need a t-test, chi-square test, Mann-Whitney U test, correlation, or regression helper |
| matplotlib | Core plotting | When you need a poster figure and want full control over labels, axes, and export size |
| seaborn | Cleaner statistical plots built on matplotlib | When you want fast box plots, violin plots, scatter plots, and regression-style visual summaries |
| Biopython | Sequence and biological file handling | When your project uses FASTA files, sequence records, or biological database formats |
| statsmodels | More detailed statistical models | When you need regression tables, confidence intervals, or model diagnostics beyond SciPy basics |
| scikit-learn | Machine learning | Only when prediction is truly part of the project; not needed for basic EBS statistics |
Teacher note: If you are new, do not start with machine learning. Most strong beginner projects use pandas, NumPy, SciPy, and one clear figure.
Most Useful pandas Commands
| Command | What it does | Example |
|---|---|---|
pd.read_csv() | Loads a CSV file into a DataFrame | df = pd.read_csv("enzyme_activity.csv") |
df.head() | Shows the first rows so you can inspect the table | df.head() |
df.shape | Reports row and column count | df.shape |
df.columns | Lists column names | df.columns |
df.info() | Shows column types and missing-value counts | df.info() |
df.describe() | Summarizes numeric columns | df.describe() |
df["column"] | Selects one column | df["enzyme_activity"] |
df.loc[] | Selects rows and columns by label or condition | df.loc[df["condition"] == "control"] |
df.query() | Filters rows using a readable expression | df.query("temperature_c == 39") |
df.value_counts() | Counts categories or repeated values | df["condition"].value_counts() |
df.isna().sum() | Counts missing values by column | df.isna().sum() |
df.dropna() | Removes rows with missing data | df.dropna(subset=["enzyme_activity"]) |
df.fillna() | Fills missing values when justified | df["ph"].fillna(df["ph"].median()) |
df.rename() | Renames confusing columns | df.rename(columns={"ph": "pH"}) |
df.astype() | Changes a column type | df["temperature_c"].astype(float) |
df.groupby() | Splits data into groups before summarizing | df.groupby("condition")["enzyme_activity"].mean() |
.agg() | Runs several summaries at once | .agg(["count", "mean", "std"]) |
df.sort_values() | Sorts rows | df.sort_values("enzyme_activity") |
pd.merge() | Combines two tables by a shared key | pd.merge(samples, metadata, on="sample_id") |
df.to_csv() | Saves a cleaned table | clean.to_csv("cleaned.csv", index=False) |
Most Useful NumPy Commands
| Command | What it does | Example |
|---|---|---|
np.array() | Creates a numeric array | values = np.array([11.2, 10.8, 11.5]) |
np.mean() | Calculates the mean | np.mean(control) |
np.median() | Calculates the median | np.median(control) |
np.std() | Calculates standard deviation | np.std(control, ddof=1) |
np.min() / np.max() | Finds smallest and largest values | np.max(treatment) |
np.percentile() | Finds percentile cutoffs | np.percentile(control, [25, 50, 75]) |
np.where() | Creates conditional values | np.where(values > 12, "high", "low") |
np.isnan() | Detects missing numeric values | np.isnan(values) |
np.unique() | Finds unique values and counts | np.unique(groups, return_counts=True) |
np.corrcoef() | Calculates a correlation matrix | np.corrcoef(temp, activity) |
np.random.seed() | Makes random work reproducible | np.random.seed(42) |
Most Useful SciPy Commands
| Command | What it does | Example |
|---|---|---|
stats.ttest_ind() | Compares two independent group means | stats.ttest_ind(control, treatment, equal_var=False) |
stats.ttest_rel() | Compares paired measurements | stats.ttest_rel(before, after) |
stats.mannwhitneyu() | Compares two groups without assuming normality | stats.mannwhitneyu(control, treatment) |
stats.chi2_contingency() | Tests association between categorical variables | stats.chi2_contingency(table) |
stats.fisher_exact() | Tests small 2x2 categorical tables | stats.fisher_exact(table) |
stats.pearsonr() | Measures linear correlation between two numeric variables | stats.pearsonr(temp, activity) |
stats.spearmanr() | Measures rank correlation | stats.spearmanr(dose, response) |
stats.linregress() | Fits a simple linear regression | stats.linregress(temp, activity) |
stats.shapiro() | Checks normality for small samples | stats.shapiro(control) |
stats.zscore() | Standardizes values or flags extreme values | stats.zscore(df["enzyme_activity"]) |
Decision rule: Use a t-test for two numeric groups, chi-square or Fisher's exact test for categorical counts, correlation for two numeric variables, and regression when one numeric variable is being used to predict another.
Practice Dataset: Enzyme Activity
This invented dataset simulates a simple biology question: does a treatment condition increase enzyme activity compared with a control condition?
| sample_id | condition | temperature_c | ph | enzyme_activity | cell_viability |
|---|---|---|---|---|---|
| E01 | control | 37 | 7.2 | 11.2 | 96 |
| E02 | control | 37 | 7.2 | 10.8 | 95 |
| E03 | control | 37 | 7.1 | 11.5 | 94 |
| E04 | control | 37 | 7.3 | 10.9 | 97 |
| E05 | control | 37 | 7.2 | 11.0 | 96 |
| E06 | control | 37 | 7.1 | 11.3 | 95 |
| E07 | treatment | 39 | 7.2 | 14.1 | 93 |
| E08 | treatment | 39 | 7.2 | 13.7 | 92 |
| E09 | treatment | 39 | 7.1 | 14.5 | 91 |
| E10 | treatment | 39 | 7.3 | 15.0 | 90 |
| E11 | treatment | 39 | 7.2 | 13.9 | 92 |
| E12 | treatment | 39 | 7.1 | 14.3 | 93 |
sample_id,condition,temperature_c,ph,enzyme_activity,cell_viability
E01,control,37,7.2,11.2,96
E02,control,37,7.2,10.8,95
E03,control,37,7.1,11.5,94
E04,control,37,7.3,10.9,97
E05,control,37,7.2,11.0,96
E06,control,37,7.1,11.3,95
E07,treatment,39,7.2,14.1,93
E08,treatment,39,7.2,13.7,92
E09,treatment,39,7.1,14.5,91
E10,treatment,39,7.3,15.0,90
E11,treatment,39,7.2,13.9,92
E12,treatment,39,7.1,14.3,93pandas: Read, Inspect, Filter, Summarize
A DataFrame is a table where each row is one observation and each column is one variable. For this dataset, each row is one enzyme sample.
Core Commands
import pandas as pd
df = pd.read_csv("enzyme_activity.csv")
print(df.head())
print(df.shape)
print(df.columns)
print(df["condition"].value_counts())
clean = df.dropna(subset=["condition", "enzyme_activity"])
summary = clean.groupby("condition")["enzyme_activity"].agg(["count", "mean", "std"])
print(summary)The important line is groupby("condition"). It tells pandas to split the table into control rows and treatment rows before calculating count, mean, and standard deviation.
NumPy: Numeric Thinking
NumPy is useful when the analysis becomes more mathematical. It treats columns as arrays of numbers, which makes calculations predictable and fast.
Mean, Difference, and Standard Deviation
import numpy as np
control = clean.loc[clean["condition"] == "control", "enzyme_activity"].to_numpy()
treatment = clean.loc[clean["condition"] == "treatment", "enzyme_activity"].to_numpy()
control_mean = np.mean(control)
treatment_mean = np.mean(treatment)
difference = treatment_mean - control_mean
print(control_mean)
print(treatment_mean)
print(difference)
print(np.std(control, ddof=1))
print(np.std(treatment, ddof=1))Teacher note: Use ddof=1 for a sample standard deviation. Biology projects almost always measure samples, not every possible organism or molecule in the world.
SciPy: Statistical Tests
The treatment mean is higher, but statistics asks whether the difference is large relative to the variation inside each group.
Welch Two-Sample t-Test
from scipy import stats
test = stats.ttest_ind(control, treatment, equal_var=False)
print(test.statistic)
print(test.pvalue)Use equal_var=False because it does not assume the two groups have exactly the same variance. This is called Welch's t-test.
Exercises
Skill Check A
- How many rows and columns are in the dataset?
- What is the mean enzyme activity for each condition?
- What is the treatment minus control difference?
Skill Check B
- What is the mean cell viability for each condition?
- Why should the lower treatment viability be mentioned as a limitation?
- Write one figure caption for a bar chart of enzyme activity by condition.
Check Your Work
Skill Check A Answers
The dataset has 12 rows and 6 columns. The control mean enzyme activity is about 11.12. The treatment mean is 14.25. The treatment group is higher by about 3.13 activity units.
Statistical Test Answer
A Welch t-test gives t = -14.39 and p = 5.79e-07 when the comparison is control minus treatment. The negative sign only reflects the order of subtraction. The practical interpretation is that treatment samples show much higher enzyme activity in this dataset.
Skill Check B Answers
Mean cell viability is 95.5 for control and 91.83 for treatment. That matters because a treatment that increases enzyme activity might also reduce cell viability. A fair poster would say: "Treatment samples showed higher enzyme activity than controls, but treatment samples also had lower cell viability, so follow-up work should test whether the activity change is independent of stress or toxicity."