Using .env files for Python Environment Variables
As a modern application, yours always deals with credentials, secrets, and configurations to connect to other services like an authentication service, database, cloud services, microservices, etc. Keeping your username, password, and other login information in the source code of your
application is not a good idea because they could be exposed if you share or publish the programme.
Before sharing the code, you must delete or remark the credentials, which requires more work from you. Also, you might eventually forget to do it. The credentials you provide to the application via the command line parameters may be visible to others as well as to you.
The source code should not also contain hard codes for the setups of the services, such as API endpoints and database URLs. The issue is that you must rewrite the code each time you change or update the configurations, which could result in further mistakes.
How should we solve this issue using environment variables?
Let’s deal with it.
In this article, we will learn about
What are environment variables?
Use cases of environment variables?
Types of environment variables
How can we set and get environment variables in Python and so on?
What are environment variables?
The variable that is connected to an environment is referred to as an environment variable. In other words, the external variables you keep outside of your program will affect how your application behaves.
Environment variables are variables that are dynamically available to your programme or application while it is running. Values of these variables may originate from text files, outside secret managers, calling scripts, etc.
The most important fact is the value of these environment variables is not hardcoded into your application. They are genuinely dynamic and adaptable to the setting in which your software is running.
Real-time example:
When I use my computer to work, it is my environment. It has a few variables that I may utilise on my machine globally. If you use Windows, you can search for environment variables in system properties and a list of them will then show.
It contains user and system environment variables and key(variable name) value(variable value) pairs of all the environment variables. Here, the PATH is a general environment variable used by OS, specifying a set of directories where executable programs are located.
Use cases of Environment variables
It is used to store configuration information and application secrets, which your running application can access as needed.
Enhance security procedures - It safeguards private data. When there are parts of your code that you don't want others to see, like API tokens that give access to your accounts you only need to call the environment variables. If the API token is stored as an environment variable rather than directly typing it out in the file, users will be able to use your code without having to hardcode their API tokens.
The usage of environment variables also eliminates the need to update your source code if your secrets change.
Without making any code modifications, you can deploy your application in any environment and ensures that sensitive information, such as API keys, is not leaking into the source code.
When configuring your Python programme, environment variables are a wonderful method to do it without having to update the source code. Third-party API keys, network ports, database servers, and any other unique choices that your application may require to function properly are common configuration elements that are frequently supplied to applications using environment variables.
You can avoid manually updating your environment variables. For example, when running a script on someone else's computer, you don't need to manually change the "user" portion of the path; instead, you can use an environment variable that returns the user's name. This means that when utilising a script on a different system, you no longer need to remember to modify that.
You can do development deployment or stage in deployment or even production deployment with environment variables. Environment variables will keep all the secrets safe.
Types of environment variables
A Windows-based computer system uses three different sorts of environment variables, each of which has specific applications. Let's talk about each one separately.
System environment variables
The system's topmost root contains system environment variables, which are shared by all processes executing in the system across all user profiles. They are typically configured by your operating system or system administrator, and you rarely need to mess with them.
One of the most popular uses of a system environment variable is configuring a PATH variable to a global package or library that will be utilised by all users of the system.
User environment variables
In Windows systems, user environment variables are those that are exclusive to a user profile. The values of programmes installed solely for particular users, the route to a local installation of libraries that are not to be utilised by all users, and other user-specific data are stored in these variables.
Changes to these variables can be made by users without the assistance of the system administrator. For making local changes to your system without affecting other users, these variables are useful.
Run-time/Process environment variables
Runtime environment variables are further restricted only to the runtime or the process that they are associated with. They are typically established by the parent process that initiates the new process, together with the user and system environment variables.
To generate and store these variables instantly, utilise terminal scripting. Unless scripted, runtime variables are often not permanent, thus you must define them each time a new process is launched.
What is a .env file?
A text file called a ".env file" stores key-value pairs for each environment variable your application needs. You can use environment variables for local development using a .env file without contaminating the global environment namespace.
It's an easy approach to save global variables that you need but doesn't want publicly available online such as storing private data. For example:
Database login information
Username and password
API tokens
Secret codes
Crypto Wallet keys
This is an example of a typical .env file:
VAR_UNO=SOME_KEY_HERE
VAR_DOS=SOME_OTHER_KEY_HERE
Here, VAR_UNO,VAR_DOS are keys and SOME_KEY_HERE, SOME_OTHER_KEY_HERE are values.
What is Python-dotenv?
It is a Python library that reads environment variables from the .env file.
Accessing environment variables from Python
os. environ dictionary
In Python, all environment variables are stored in the os. environ dictionary. The use of this conventional dictionary syntax is the simplest method for retrieving a variable from within your programme. You can use this method, for instance, to obtain the environment variable USER:
>>> import os
>>> user = os.environ['USER']
>>> user
'miguel'
With this approach, Python will throw a KeyError exception if you attempt to import a nonexistent environment variable:
>>> database_url = os.environ['DATABASE_URL']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/miguel/.pyenv/versions/3.8.6/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'DATABASE_URL'
How to get an environment variable in Python
The os.environ object in Python is used to retrieve environment variables. Although the os.environ object resembles a dictionary, it differs because only strings may be used as values, and they cannot be serialised to JSON. When it comes to referencing the os.environ object, you have a few choices:
# 1. Standard way
import os
# os.environ['VAR_NAME']
# 2. Import just the environ object
from os import environ
# environ['VAR_NAME']
# 3. Rename the `environ` to env object for more concise code
from os import environ as env
# env['VAR_NAME']
Depending on what should occur if an environment variable is missing, there are three different ways to access a given environment variable in Python.
Let’s experience this with examples,
Required with no default value
Get the environment variable directly if your app crashes when it is not set:
print(os.environ['HOME']
# >> '/home/dev'
print(os.environ['DOES_NOT_EXIST']
# >> Will raise a KeyError exception
For instance, the application shouldn't be able to launch if a key environment variable is not declared and a default value, such as a database password, cannot be supplied. Instead of the typical KeyError exception (which doesn't explain why your app didn't launch), if you could catch the KeyError exception and display a helpful message instead, then:
import os
import sys
# Ensure all required environment variables are set
try:
os.environ['API_KEY']
except KeyError:
print('[error]: `API_KEY` environment variable required')
sys.exit(1)
Required with a default value
Using the os.environ.get method and passing the default value as the second parameter, you can retrieve a default value back if an environment variable is missing:
# If HOSTNAME doesn't exist, presume local development and return localhost
print(os.environ.get('HOSTNAME', 'localhost')
None is returned if the variable doesn't exist and you call os.environ.get without specifying a default value.
assert os.environ.get('NO_VAR_EXISTS') == None
Conditional logic if the value exists
Although you might want to know if an environment variable exists, you don't necessarily need to know its value. For instance, If the DEBUG environment variable is set, your application may enter "Debug mode."
You can confirm an environment variable's existence by itself:
if 'DEBUG' in os.environ:
print('[info]: app is running in debug mode')
Or see whether it matches a certain value:
if os.environ.get('DEBUG') == 'True':
print('[info]: app is running in debug mode')
How to set an environment variable in Python
In Python, setting an environment variable is equivalent to setting a dictionary's key:
os.environ['TESTING'] = 'true'
The only permitted values in os.environ are strings, which distinguishes it from a typical dictionary:
os.environ['TESTING'] = True
# >> TypeError: str expected, not bool
Although there are use cases for setting them as well, your application will typically just need to get environment variables.
For example, constructing a DB_URL environment variable on application start-up using DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, and DB_NAME environment variables:
os.environ['DB_URL'] = 'psql://{user}:{password}@{host}:{port}/{name}'.format(
user=os.environ['DB_USER'],
password=os.environ['DB_PASSWORD'],
host=os.environ['DB_HOST'],
port=os.environ['DB_PORT'],
name=os.environ['DB_NAME']
)
Look at another illustration of setting a default value of a variable following the value of another variable:
# Set DEBUG and TESTING to 'True' if ENV is 'development'
if os.environ.get('ENV') == 'development':
os.environ.setdefault('DEBUG', 'True') # Only set to True if DEBUG not set
os.environ.setdefault('TESTING', 'True') # Only set to True if TESTING not set
How to delete an environment variable in Python
Use the os.environ.pop function to remove a Python environment variable. For example, you might want to remove the other DB_ prefixed fields to make sure that the app can only access the database through the DB _URL that we previously gave.
Let’s see an additional example of removing an environment variable when it is no longer required:
auth_api(os.environ['API_KEY']) # Use API_KEY
os.environ.pop('API_KEY') # Delete API_KEY as it's no longer needed
List the Python environment variables
To see every environment variable in Python, you could use the os.environ object. It denotes environment variables with their values. It provides a dictionary with the environmental variable names as the “keys” and their values as the “values”.
In the following programme, we loop through the dictionary returned by the os.environ. The key represents the name of the environment variables, and the value represents the value of the environment variables:
import os
for name, value in os.environ.items():
print("{0}: {1}".format(name, value))
Output:
Using a .env file for environment variables in Python
The number of environmental variables increases along with the size and complexity of a programme. Most of the projects experience growing difficulties when using environment variables for application configuration and secrets because there is a lack of clear and consistent strategy for how to manage them, especially when deploying to multiple environments.
A simple and better solution is to use a .env file to include all of the variables for a particular environment and use a Python library such as python-dotenv to parse the .env file and populate the os.environ object.
Install the python-dotenv library after creating and activating a new virtual environment to follow along:
# 1. Create
python3 -m venv ~/.virtualenvs/doppler-tutorial
# 2. Activate
source ~/.virtualenvs/doppler-tutorial/bin/activate
# 3. Install dotenv package
pip install python-dotenv
Save the following to a file with the extension ".env" (notice that it uses the same syntax as setting a variable in the shell):
API_KEY="357A70FF-BFAA-4C6A-8289-9831DDFB2D3D"
HOSTNAME="0.0.0.0"
PORT="8080"
Next, save the subsequent code to dotenv-test.py:
# Rename `os.environ` to `env` for nicer code
from os import environ as env
from dotenv import load_dotenv
load_dotenv()
print('API_KEY: {}'.format(env['API_KEY']))
print('HOSTNAME: {}'.format(env['HOSTNAME']))
print('PORT: {}'.format(env['PORT']))
After that, launch dotenv-test.py to verify that the environment variables are populating:
python3 dotenv-test.py
# >> API_KEY: 357A70FF-BFAA-4C6A-8289-9831DDFB2D3D
# >> HOSTNAME: 0.0.0.0
# >> PORT: 8080
Final thoughts
In this blog, we looked at how to use the Python dotenv package and an.env file to manage environment variables.
Environment variables play a vital role to isolate sensitive data from your application. They assist in keeping your app's secrets safe and make it simple for you to switch between sets of secrets based on the circumstances. But taking care of them adds another duty to your plate.
Your client ids, hostnames, database names, port numbers, secrets, and other data are frequently stored in environment variables in the form of key-value pairs to keep them safe.
python-dotenv can be a very helpful tool for handling various environment variables, sensitive data and configuration choices in your Python applications and helps to avoid hard-coding sensitive information into your code.
Environment variables files are text files with the ".env" extension that can be used to store environmental variables. It's a quick and easy approach to save global variables that you want to utilise but don't want to be accessible to the public.
Comments
Post a Comment