Data Science: How to — Write CSV from an API (JSON)

  1. The reposity of this post
  2. My Github
  3. For all my posts

How To

In this “How To” series I share part of the knowledge I learned, with the intention of strengthen the Data Science community and create free and useful content for everyone.


  1. import requests
  2. from time import sleep
  3. import pandas as pd
  4. import numpy as np

Selecting an API provide an API for free, with a limit of 1 request per second.

def api_json():
#link of the API
link = ''

#retrieving the JSON
json = requests.get(link).json()

#getting columns and values from the JSON
columns = json.keys()
values = json.values()

#returning the JSON, the columns and the values
return json, columns, values

The Function is already done, now let’s assign the returning values from the Function:

#calling the api_json() and assigning it to 3 variables
api, columns, values = api_json()

Creating the CSV and Writing the columns

This Function, uses the number of columns in the JSON file to set when comma is set, for separating the values, and when line break is set for a new row to be written:

def create_dataset():
database = open("dataset.txt", "a")
counter = 0
for column in columns:
if counter == len(columns) - 1:
database.write(column + '\n')
database.write(column + ',')
counter = counter + 1

Inserting New Values to the CSV file

The Function below asks us just two variables,

  1. n_rows: the number of rows, you want in your data base
  2. interval_seconds: interval in seconds that you want to retrieve data from the API.
def writing_values(n_rows, interval_seconds):
counter = 0
for i in range(n_rows):
api, columns, values = api_json()
for value in values:
database = open("dataset.txt", "a")
if counter == len(values) - 1:
database.write(str(value) + '\n')
database.write(str(value) + ',')
counter = counter + 1 counter = 0 #restarting the counter for new rows

Let’s check if it’s running

create_dataset() #creates the dataset

The code above just create the “dataset.txt” and write the columns.

The code beloew writes rows in the file according to the pre-settled variables:

writing_values(5, 3) #5 rows, using 3 seconds as interval

The Result

df = pd.read_csv('dataset.txt')

Hope I helped you learn something new.

Igor Comune.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Igor Comune

Igor Comune

An "under construction" Data Scientist with a love for numbers and analysis!