Technography

  Web Scraping in Node.js using Puppeteer Table of contents Introduction What is Web Scraping? What is a Web Crawler? Why should we use Web Scraping with Node.js? Web Scraping in Node.js using Puppeteer? Use Cases of Web Scraping Pros & Cons of Web Scraping Conclusion Introduction In this digital era, data plays a vital role in every process. From sales to machine learning, the functionality of every process is data-centric. Various types of data namely text, image, audio, and video are used depending on the process.  Collecting these types of data from various resources for a particular process is called data collection. Nowadays data collection is done via the internet manually or programmatically depending upon the process. For a small data process manual data collection is enough but for a large data process such as training an AI bot requires a huge amount of data, in this case, manual data collection will be a very difficult and more time-consuming process. To avoid ...

  Using .env files for Python Environment Variables As a modern application, yours always deals with credentials, secrets, and configurations to connect to other services like an authentication service, database, cloud services, microservices, etc. Keeping your username, password, and other login information in the source code of your  application is not a good idea because they could be exposed if you share or publish the programme.  Before sharing the code, you must delete or remark the credentials, which requires more work from you. Also, you might eventually forget to do it. The credentials you provide to the application via the command line parameters may be visible to others as well as to you. The source code should not also contain hard codes for the setups of the services, such as API endpoints and database URLs. The issue is that you must rewrite the code each time you change or update the configurations, which could result in further mistakes. How should we solv...