User Prompt

Task: Python script to merge CSVs
Input: Multiple CSVs, same columns
Constraints: Pandas only, <50 lines
Output: Single merged.csv
Verify: Run on test_data/

General Code Structure

The current structure is an index.html and index.tsx with an es6 module that is automatically imported by the index.html.

Sample Code for Merging CSVs

Here’s a simple Python script using Pandas to merge multiple CSV files:

import pandas as pd
import glob

# glob to fetch all csv files
path = "test_data/"
all_files = glob.glob(path + "*.csv")

# list to hold dataframes
dfs = []

# loop through all files and read them into a dataframe
for file in all_files:
    df = pd.read_csv(file)
    dfs.append(df)

# merge all dataframes into a single dataframe
merged_df = pd.concat(dfs, ignore_index=True)

# write the merged dataframe to a new csv file
merged_df.to_csv("merged.csv", index=False)
print("CSV files have been merged successfully!")

Explanation of the Code

  1. Import Modules: The pandas and glob modules are imported. pandas is used for data manipulation and analysis, while glob is used to retrieve files/pathnames matching a specified pattern.
  2. File Path: Define the path where the CSV files are stored.
  3. Retrieve Files: Use glob.glob() to get a list of all CSV files in the specified directory.
  4. Read and Append Dataframes: Loop through the list of files, read each file into a DataFrame, and append it to a list.
  5. Merge Dataframes: Use pd.concat() to combine the list of DataFrames into a single DataFrame.
  6. Save Merged CSV: Write the combined DataFrame to a new CSV file.

Testing the Script

To confirm that the merging function works correctly, run the script in an environment where you have a folder named test_data containing multiple CSV files with the same columns. Check that the merged.csv file is created with all entries combined correctly.