Automatically generate a PynamoDB model and CRUD functions for Amazon DynamoDB by using a Python application
Created by Vijit Vashishtha (AWS), Dheeraj Alimchandani (AWS), and Dhananjay Karanjkar (AWS)
Summary
It's common to require entities and create, read, update, and delete (CRUD) operations functions to efficiently perform Amazon DynamoDB database operations. PynamoDB is a Python-based interface that supports Python 3. It also provides features such as support for Amazon DynamoDB transactions, automatic attribute value serialization and deserialization, and compatibility with common Python frameworks, such as Flask and Django. This pattern helps developers working with Python and DynamoDB by providing a library that streamlines the automatic creation of PynamoDB models and CRUD operation functions. While it generates essential CRUD functions for database tables, it can also reverse engineer PynamoDB models and CRUD functions from Amazon DynamoDB tables. This pattern is designed to simplify database operations by using a Python-based application.
The following are the key features of this solution:
JSON schema to PynamoDB model – Automatically generate PynamoDB models in Python by importing a JSON schema file.
CRUD function generation – Automatically generate functions to perform CRUD operations on DynamoDB tables.
Reverse engineering from DynamoDB – Use PynamoDB object-relational mapping (ORM) to reverse engineer PynamoDB models and CRUD functions for existing Amazon DynamoDB tables.
Prerequisites and limitations
Prerequisites
An active AWS account
Python version 3.8 or later, downloaded
and installed Jinja2 version 3.1.2 or later, downloaded
and installed Amazon DynamoDB tables for which you want to generate ORM
AWS Command Line Interface (AWS CLI), installed and configured
PynamoDB version 5.4.1 or later, installed
Architecture
Target technology stack
JSON script
Python application
PynamoDB model
Amazon DynamoDB database instance
Target architecture
You create an input JSON schema file. This JSON schema file represents the attributes of the respective DynamoDB tables that you want to create PynamoDB models from and CRUD functions for. It contains the following three important keys:
name–The name of the target DynamoDB table.region– The AWS Region where the table is hostedattributes– The attributes that are part of the target table, such as the partition key (also known as a hash attribute), sort key, local secondary indexes, global secondary indexes, and any non-key attributes. This tool expects the input schema to only provide the non-key attributes as the application fetches the key attributes directly from the target table. For an example of how to specify attributes in the JSON schema file, see the Additional information section of this pattern.
Run the Python application and provide the JSON schema file as an input.
The Python application reads the JSON schema file.
The Python application connects to the DynamoDB tables to derive the schema and data types. The application runs the describe_table
operation and fetches the key and index attributes for the table. The Python application combines the attributes from the JSON schema file and DynamoDB table. It uses the Jinja template engine to generate a PynamoDB model and corresponding CRUD functions.
You access the PynamoDB model to perform CRUD operations on the DynamoDB table.
Tools
AWS services
Amazon DynamoDB is a fully managed NoSQL database service that provides fast, predictable, and scalable performance.
Other tools
Jinja
is an extensible templating engine that compiles templates into optimized Python code. This pattern uses Jinja to to generate dynamic content by embedding placeholders and logic within templates. PynamoDB
is a Python-based interface for Amazon DynamoDB. Python
is a general-purpose computer programming language.
Code repository
The code for this pattern is available in the GitHub Auto-generate PynamoDB models and CRUD functions
Controller package
The controller Python package contains the main application logic that helps generate the PynamoDB model and the CRUD functions. It contains the following:
input_json_validator.py– This Python scripts validates the input JSON schema file and creates the Python objects that contain the list of target DynamoDB tables and the required attributes for each.dynamo_connection.py– This script establishes a connection to the DynamoDB table and uses thedescribe_tableoperation to extract the attributes that are necessary to create the PynamoDB model.generate_model.py– This script contains a Python classGenerateModelthat creates the PynamoDB model based on the input JSON schema file and thedescribe_tableoperation.generate_crud.py– For the DynamoDB tables that are defined in the JSON schema file, this script uses theGenerateCrudoperation to create the Python classes.
Templates
This Python directory contains the following Jinja templates:
model.jinja– This Jinja template contains the template expression for generating the PynamoDB model script.crud.jinja– This Jinja template contains the template expression for generating the CRUD functions script.
Epics
| Task | Description | Skills required |
|---|---|---|
Clone the repository. | Enter the following command to clone the Auto-generate PynamoDB models and CRUD functions | App developer |
Set up the Python environment. |
| App developer |
| Task | Description | Skills required |
|---|---|---|
Modify the JSON schema file. |
| App developer |
Run the Python application. | Enter the following command to generate the PynamoDB models and CRUD functions, where | App developer |
| Task | Description | Skills required |
|---|---|---|
Verify the generated PynamoDB model. |
| App developer |
Verify the generated CRUD functions. |
| App developer |
Related resources
Core components of Amazon DynamoDB (DynamoDB documentation)
Improving data access with secondary indexes (DynamoDB documentation)
Additional information
Sample attributes for the JSON schema file
[
{
"name": "test_table",
"region": "ap-south-1",
"attributes": [
{
"name": "id",
"type": "UnicodeAttribute"
},
{
"name": "name",
"type": "UnicodeAttribute"
},
{
"name": "age",
"type": "NumberAttribute"
}
]
}
]