Python 3 Github API Web Scraper or Crawler Script to Scrape User Profile and Repositories Details Using requests and json Modules Full Project For Beginners

You are currently viewing Python 3 Github API Web Scraper or Crawler Script to Scrape User Profile and Repositories Details Using requests and json Modules Full Project For Beginners

 

Welcome folks today in this post we will be building a github web scraper in python using json module. All the full source code of application will be given below.

 

 

 

Features of Scraper

 

 

It will scrape user details and repositories information from github

 

It will store this information into a text file

 

 

Get Started

 

 

In order to get started you need to make an app.py file and copy paste the following code

 

app.py

 

 

 

 

Now in order to run this python script you need to provide an additional command line argument which will be the actual github username of the user

 

python app.py ##githubusername##

 

For my case this will be

 

python app.py gauti123456

 

So after execution this script will create a text file in the same root directory and in that file all the information regarding the user will be there as shown below

 

 

 

Leave a Reply