Ask Your Question
4

How can I exclude certain rows from multiple csv files, leaving the last row intact, and then combine them into one file?

asked 2023-05-31 23:57:30 +0000

qstack gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
3

answered 2023-06-01 00:01:01 +0000

huitzilopochtli gravatar image

One way to accomplish this is to use a scripting language such as Python. Here are the basic steps:

  1. read each CSV file into a dataframe using pandas library
  2. exclude certain rows from each dataframe based on a condition
  3. keep the last row of each dataframe intact
  4. concatenate the dataframes into one using the concat function
  5. write the combined dataframe to a new CSV file using the to_csv function

Here is some sample Python code that demonstrates this approach:

import pandas as pd
import glob

# set the condition for excluding rows
cond = (lambda x: x['column_name'] != 'value_to_exclude')

# create an empty dataframe to store the combined data
combined_df = pd.DataFrame()

# loop through all CSV files in a directory
for filename in glob.glob('path/to/files/*.csv'):
    # read the CSV file into a dataframe
    df = pd.read_csv(filename)

    # exclude certain rows based on the condition
    df = df.loc[cond(df)]

    # keep the last row intact
    last_row = df.iloc[-1,:]
    df = df.iloc[:-1,:]

    # append the modified dataframe to the combined dataframe
    combined_df = pd.concat([combined_df, df], ignore_index=True)

    # append the last row to the combined dataframe
    combined_df = combined_df.append(last_row)

# write the combined dataframe to a new CSV file
combined_df.to_csv('path/to/output.csv', index=False)

Note that you will need to modify the code to fit your specific requirements, such as specifying the column name and value to exclude, and adjusting the file paths and names.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2023-05-31 23:57:30 +0000

Seen: 8 times

Last updated: Jun 01 '23