Building an AirBnB Clone: The Console 2
Stepping up - File Storage 📂
Introduction
In the previous article we created various class models that represented various parts of the data which we would want to store.
In this article, we would look at how we can actually store these models using a file storage as discussed in the previous article
Building the File Storage Engine
Now we are able to create an instance using the dictionary which is great because it keeps track of all our instances. But there is a problem: every time we restart the program, we would have to recreate everything from scratch.
To solve that, we need to find a way to save all the data so that when we restart the program, it could just be reloaded and then we can add new instances to our previous ones.
Hence, in building our data storage, we need to ensure persistency.
Persistency refers to the ability of data to remain stored and accessible even after the program or device is shut down or restarted. It ensures that information remains intact over time, allowing users to retrieve and utilize it whenever needed. Think of it like saving a document on your computer—once saved, the document stays there until you decide to delete or modify it.
How to ensure data persistency
Here is a proposed solution to that
We would simply need to write the objects to a file and save it.
This would mean, converting the instance to a dictionary and then writing it into a file in JSON format to be saved.
Now you might ask why we should use this approach. Well here are the reasons
Python doesn't know how to convert a string into a dictionary easily.
The dictionary isn't human readable.
Using the dictionary in another language might be a bit difficult since different languages have different ways of implementing dictionaries.
JSON in Simple Terms:
JSON, short for JavaScript Object Notation, is a lightweight data format used for storing and exchanging information between different systems. It resembles a structured text format, making it easy to read and write by both humans and computers. It's like packaging up data in a neat box so it can be easily shared and understood by different programs.
So, the process would look like this diagrammatically.
<class 'BaseModel'> -> to_dict() -> <class 'dict'> -> JSON dump -> <class 'str'> -> FILE -> <class 'str'> -> JSON load -> <class 'dict'> -> <class 'BaseModel'>
And more like this in a pictorial view
Read here for the difference between the dictionary and JSON
Storing the Objects
From our diagram, the steps for serialization (storage to the file) would be as follows:
Convert the instance to dictionary using
to_dict().Convert the dictionary to JSON using a method known as
json.dump(s).This would produce a pure string and then we save it to a FILE.
The steps for deserialization (retrieval from the file) would be as follows:
Open the FILE which has the JSON object.
Retrieve the JSON using the method
json.load(s).Convert this to a dictionary and use that dictionary to create an instance.
Creating the Engine Folder
We would create a separate folder to handle this entire process. This folder would be in our "models" folder.
mkdir engine
Setting up the File Storage Class
We would set up a class to handle the serialization and deserialization. This class would also have other attributes which would come in handy for our usage. In the engine folder, create two files __init__.py and file_storage.py
cd engine
touch __init__.py file_storage.py
NOTE: The presence of an "init" file makes the "engine" folder a package.
Role of the File Storage Class
Our FileStorage class would be responsible for serializing instances to JSON and deserializing JSON files to instances.
Now let's build our FileStorage Class 🔥
The FileStorage Class
To build out the class attributes and various methods we need to know what exactly we want. I created a list for that.
List of things to be done
We want to have an objects dictionary, which stores all the instances.
We need a way to save these objects in the dictionary into a file (serialize)
From point two, we would have to create/get the file which would serve as the storage for now.
We would figure out how to add a new instance object to the dictionary of objects.
A function would be needed to retrieve all these instances in the dictionary of objects.
A function to deserialize the data from the file. This would be very crucial as we want to ensure persistency. When the program relaunches, the dictionaries in the file are simply used to recreate all the instances before a user begins to interact with the system again.
From the above we would have to create the following attributes and methods
Attributes
__fiile_path: This would be a string to store our file path (ex. "file.json").__objects: This would help us with the dictionary of objects spoken about earlier. We would store all the objects in this regard<classname>.id.Example, to store a
Userobject withid = 321423, the key for this entry would beUser.321423. We just want to have a way to access each instance by their names and id (which is unique). This would be very handy as two instances would always have different ids.
Methods
all(self): This would return the dictionary of objects ie.__objects. (See point 5 of the lists I created).new(self, obj): This adds a new created instance to the dictionary of objects ie.__objects(See point 4 of the list). The key for this would be<obj classname>.idjust as we saw in the attributes above.save(self): This method would simply save or serialize all the items in the dictionary of objects to the json file ie__file_path. This fulfils point 2.reload(self): As explained in point 6 of our list, we would deserialize (convert the JSON to a dictionary) the JSON file. In this function, we would that data to recreate all the instances for every session which the user launches the program. This would make our storage much persistent, as input stored is always regenerated and new inputs added are stored in the dictionary of objects and then subsequently stored in the file.
Let's start coding 🔥
Create FileStorage class and add the Attributes of the Class
In your file_storage.py file add these lines of code
#!/usr/bin/python3
"""
Module for file storage
"""
import json
class FileStorage:
"""
- Serializes instances to JSON
- Deserializes JSON to instances
"""
__file_path = "file.json"
__objects = {}
NOTE: Remember to import the JSON module as we would be using it in the file for both serialization and deserialization
All() method
def all(self):
"""Returns dictionary of objects"""
return self.__objects
new(self, obj) method
def new(self, obj):
"""
Sets an object in __objects with key <obj_class>.id
"""
key = "{}.{}".format(obj.__class__.__name__, obj.id)
self.__objects[key] = obj
Analysis
This takes an object (an instance) as an argument.
obj.__class__.__name__: This retrieves the classname of the object.obj.id: This gives us the id of the object.Since we are simply populating the
__objectsdictionary, we do so with a key and value pair. Our key would be the<obj_class>.idand the value would be the entireobjwhich has been passed.
save(self) method
Here's a recap of the diagram for serialization
<class 'BaseModel'> -> to_dict() -> <class 'dict'> -> JSON dump -> <class 'str'> -> FILE
Now let's code the function
def save(self):
"""
Serializes __objects into JSON file (path __file_path)
"""
obj_dict = {}
for key, obj in self.all().items():
obj_dict[key] = obj.to_dict()
with open(self.__file_path, "w", encoding="UTF-8") as text_file:
json.dump(obj_dict, text_file)
Analysis
I created a temporary dictionary of objects (
obj_dict) and then added every item in the original__objectsto it using a for loop. You could also use the__objectsdirectly in the "with" statement.Open a file (
file.json) in write mode and then dump the dictionary in the file.
reload(self)
Here's a recap of the diagram for deserialization
FILE -> <class 'str'> -> JSON load -> <class 'dict'> -> <class 'BaseModel'>
Now let's code the function
def reload(self):
"""
Deserializes the JSON file to __objects only if JSON file exists
Otherwise do nothing. If the file doesn't exist no exception should
be raised
"""
from models.base_model import BaseModel
from models.user import User
from models.city import City
from models.place import Place
# Import others as needed and add to the calss map dictionary below
class_map = {
'BaseModel': BaseModel,
'User': User,
'City': City,
'Place': Place
}
try:
with open(self.__file_path, "r", encoding="UTF-8") as text_file:
obj_dict = json.load(text_file)
for key, val in obj_dict.items():
class_name = val['__class__']
class_instance = class_map[class_name]
instance = class_instance(**val)
all_objects = self.all()
all_objects[key] = instance
except FileNotFoundError:
pass
Analysis
We open the file in read mode. This data would be loaded and transformed into something like our
__objectsdictionary (dictionary of objects).We loop through their key and value pairs which now looks like this
{"BaseModel.b1806937-2f26-4053-aecd-ca259ea69ddd": {"id": "b1806937-2f26-4053-aecd-ca259ea69ddd", "created_at": "2024-02-09T12:06:45.830851", "updated_at": "2024-02-09T12:06:45.830855", "__class__": "BaseModel"}, "User.a2f1fa1b-b5da-49f4-9338-a0b685482db0": {"id": "a2f1fa1b-b5da-49f4-9338-a0b685482db0", "created_at": "2024-02-09T12:07:49.418902", "updated_at": "2024-02-09T12:07:49.418916", "__class__": "User"} }We set the class name as it is.
Then we get the instance of that class from our
class_mapdictionary and save in theclass_instancevariable. Theclass_mapdictionary helps us to map various string to their respective models as we imported them. This is a very nice way to handle it such that we can recreate the instances of different classes as we import them.We recreate the instance. We are imagining something like so
class_instance = class_map[class_name] # This would return let's say BaseModel instance = class_instance(**val) # this is equivalent to instance = BaseModel(**val) # where val is the dictionary of key/value pairs of the particular keyThe
instancevariable is created with the**kwargsconcept we learnt in the previous article.We call the
self.all()which would give us a list of the already listed dictionaries and we save into theall_objectsvariable.Finally, we update the
all_objectswhich would update the__objectsdictionary by adding our new instance as an entry.
Reason for importing the modules in the function
The models were imported in the function, so as to prevent what is known as Circular Import.
Importing it in the function would ensure that, it is imported onlywhen that function is called.
Congratulations, now our storage functions have been successfully created. We would now tackle how to make our other class functions utilize this storage we have provided.
Updating the __init__.py file
Anytime we fun the program, we want our reload function to be kickstarted, so we get the saved data. Then our users can interact with this data, add or remove from them. In order to achieve this, we would have to add a block of code in the models/__init__.py file. The "init" file, would run automatically anytime the program is launched.
#!/usr/bin/python3
"""
Creating a unique File Storage instance for my application
"""
from models.engine.file_storage import FileStorage
storage = FileStorage()
storage.reload()
Analysis
We import the
FileStorageclass from the "models/engine/file_storage" module.The
FileStorageclass is stored in an instance calledstorage.Using the new instance, we call the reload function to repopulate the data for our new program to be used.
Based on these changes, we would have to modify our BaseModel function as this is the function to which all the other classes would inherit.
Updating the BaseModel.py file
from models import storage # Add this to the imports
def __init__(self, *args, **kwargs):
"""
Initialiazes new instance of BaseModel.
Args:
*args: Unused positional arguments
**kwargs: Dictionary representation of an instance.
If kwargs is not empty:
Each key has an attribute name
Each value is the value of the corresponding attr name
Convert datetime to datetime objects
Otherwise:
Create id and created_at values as initially done
"""
if kwargs:
if '__class__' in kwargs:
# Remove '__class__' from the dictionary
del kwargs['__class__']
if 'created_at' in kwargs:
kwargs['created_at'] = datetime.strptime(
kwargs['created_at'], '%Y-%m-%dT%H:%M:%S.%f')
if 'updated_at' in kwargs:
kwargs['updated_at'] = datetime.strptime(
kwargs['updated_at'], '%Y-%m-%dT%H:%M:%S.%f')
for key, value in kwargs.items():
setattr(self, key, value)
else:
self.id = str(uuid.uuid4())
self.created_at = datetime.now()
self.updated_at = datetime.now()
storage.new(self) # Added this line so that our initialization is saved in storage
def save(self):
"""
Updates the public instance attribute update_at
with the current datetime
"""
self.updated_at = datetime.now()
storage.save() """ Added this line so that the save function actually calls the save function
calls the save function in the storage to serialize this in JSON file"""
Conclusion
In the upcoming article, we'll take our project one step further by diving into the creation of the console interface using cmd.cmd. This interactive component will serve as a crucial tool for testing and interacting with our file storage system, bringing our project to life.
If you found this article helpful, show your support by leaving a like 💖, following the blog 👍🏾 and sharing your thoughts in the comments below. Your feedback fuels my journey!
Follow me on GitHub, let's get interactive on Twitter and form great connections on LinkedIn 😊
Happy coding 🥂