Welcome folks today in this tutorial we will be building a simple node.js app which will download website source code files
including html,css,js and images of website from url in node.js. All the source code of the application will be given below.
Get Started
Now to get started you need to be having node.js
installed on your system. And now create a empty node.js project by initializing this command which is below
npm init -y
This will create a empty package.json file for your project
Now you need to install this dependency which will download the source code of website which is node-site-downloader
npm i node-site-downloader
This will install this package or library inside your package.json file.
Now we also need to install this globally inside your system so that the command will work globally
To install it globally we need to ad a -g
flag to the command like this
npm i -g node-site-downloader
Downloading Whole Website Files Locally
Now we can use this library to download all the website files locally inside your system. This will be full website or clone
Now let’s suppose we want to download google.com
website source code files and images we will execute the following command
Usage
1 |
node-site-downloader download DOMAIN START_POINT OUTPUT_FOLDER [VERBOSE] [OUTPUT_FOLDER_SUFFIX] [INCLUDE_IMAGES] |
Here -o
is the output folder path
--include-images
It is a option when you will apply will tell to the cli tool to also download images of the website.
Now the command will become for google.com
Make sure you add the prefix https://
at the starting of the domain if you don’t write this you will get an error
node-site-downloader download -s https://google.com -d https://google.com -v --include-images -o websitefiles
You can see that inside the command we have some options
download
It is a flag that you are passing to the cli tool that you wanna download the website. It is mandatory
--include-images
It is the option telling that you need to download the images of the website as well.
-d
It stands for domain option that you are passing
Just copy paste this command and change your website to your website path to download all the website files including the source code html, js and css and also the images of the website also.
Now inside the root directory make a index.js
file and copy paste the following code
index.js
1 2 3 4 5 6 7 8 9 10 |
const fs = require("fs"); const {exec} = require('child_process') const websiteUrl = 'https://nodejs.org/'; var directoryPath = Date.now() exec(`node-site-downloader download -s ${websiteUrl} -d ${websiteUrl} -o ${directoryPath} -v --include-images`,(err,stdout,stderr) =>{ if(err){ console.log(err) } }) |