04. CSV

CSV

Comma-Separated Values (CSV) The file is a delimited text file where values ​​are separated by commas. Each line in the file is a data record.

Each record consists of one or more fields separated by commas.

CSVLoader

• CSV Load data one row per document.

from langchain_community.document_loaders.csv_loader import CSVLoader

# Create a CSV loader
loader = CSVLoader(file_path="./data/titanic.csv")

# load data
docs = loader.load()

print(len(docs))
print(docs[0].metadata)

891
{'source': './data/titanic.csv', 'row': 0}

Customizing CSV parsing and loading

See the csv module documentation for more information on supported csv args.

# Column information:
# PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked

# CSV file path
loader = CSVLoader(
    file_path="./data/titanic.csv",
    csv_args={
        "delimiter": ",",  # separator
        "quotechar": '"',  # quote character
        "fieldnames": [
            "Passenger ID",
            "Survival (1: Survived, 0: Died)",
            "Passenger Class",
            "Name",
            "Sex",
            "Age",
            "Number of Siblings/Spouses Aboard",
            "Number of Parents/Children Aboard",
            "Ticket Number",
            "Fare",
            "Cabin",
            "Port of Embarkation",
        ],  # field name
    },
)

# load data
docs = loader.load()

# data output
print(docs[1].page_content)

Passenger ID: 1
Survival (1: Survived, 0: Died): 0
Passenger Class: 3
Name: Braund, Mr. Owen Harris
Sex: male
Age: 22
Number of Siblings/Spouses Aboard: 1
Number of Parents/Children Aboard: 0
Ticket Number: A/5 21171
Fare: 7.25
Cabin: 
Port of Embarkation: S

Use the source_column argument to specify the source of the document generated for each row. Otherwise, file_path is used as the source for all documents.

This is useful when using a chain of questions to answer questions using sources loaded from a CSV file.

loader = CSVLoader(
    file_path="./data/titanic.csv", source_column="PassengerId"
)  # Set up CSV loader, specify file path and source columns

docs = loader.load()  # load data

print(docs[1])  # data output

page_content='PassengerId: 2
Survived: 1
Pclass: 1
Name: Cumings, Mrs. John Bradley (Florence Briggs Thayer)
Sex: female
Age: 38
SibSp: 1
Parch: 0
Ticket: PC 17599
Fare: 71.2833
Cabin: C85
Embarked: C' metadata={'source': '2', 'row': 1}

UnstructuredCSVLoader

You can also load tables using UnstructuredCSVLoader. One advantage of using UnstructuredCSVLoader is that when used in "elements" mode, the metadata provides an HTML representation of the table.

from langchain_community.document_loaders.csv_loader import UnstructuredCSVLoader

# Creating an unstructured CSV loader instance
loader = UnstructuredCSVLoader(file_path="./data/titanic.csv", mode="elements")

# load document
docs = loader.load()

# Output HTML text metadata for the first document
print(docs[0].metadata["text_as_html"][:1000])

DataFrameLoader

• Output HTML text metadata for the first document

import pandas as pd

# Read CSV file
df = pd.read_csv("./data/titanic.csv")

Query the first 5 rows..

# The first five rows of the dataframe
df.head()

PassengerId	Survived	Pclass	Name	Sex	Age	SibSp	Parch	Ticket	Fare	Cabin	Embarked
0	1	0	3	Braund, Mr. Owen Harris	male	22.0	1	0	A/5 21171	7.2500	NaN	S
1	2	1	1	Cumings, Mrs. John Bradley (Florence Briggs Th...	female	38.0	1	0	PC 17599	71.2833	C85	C
2	3	1	3	Heikkinen, Miss. Laina	female	26.0	0	0	STON/O2. 3101282	7.9250	NaN	S
3	4	1	1	Futrelle, Mrs. Jacques Heath (Lily May Peel)	female	35.0	1	0	113803	53.1000	C123	S
4	5	0	3	Allen, Mr. William Henry	male	35.0	0	0	373450	8.0500	NaN	S

from langchain_community.document_loaders import DataFrameLoader

# 데이터 프레임 로더 설정, 페이지 내용 컬럼 지정
loader = DataFrameLoader(df, page_content_column="Name")

# 문서 로드
docs = loader.load()

# 데이터 출력
print(docs[0].page_content)

# 메타데이터 출력
print(docs[0].metadata)

Braund, Mr. Owen Harris
{'PassengerId': 1, 'Survived': 0, 'Pclass': 3, 'Sex': 'male', 'Age': 22.0, 'SibSp': 1, 'Parch': 0, 'Ticket': 'A/5 21171', 'Fare': 7.25, 'Cabin': nan, 'Embarked': 'S'}

# 큰 테이블에 대한 지연 로딩, 전체 테이블을 메모리에 로드하지 않음
for row in loader.lazy_load():
    print(row)
    break  # 첫 행만 출력

page_content='Braund, Mr. Owen Harris' metadata={'PassengerId': 1, 'Survived': 0, 'Pclass': 3, 'Sex': 'male', 'Age': 22.0, 'SibSp': 1, 'Parch': 0, 'Ticket': 'A/5 21171', 'Fare': 7.25, 'Cabin': nan, 'Embarked': 'S'}