Part 1 of this series was an introduction to the Tenant Service project, the AWS services used and configurations for the Serverless Framework.
In this article, part 2, we will take a deeper look at the composition of the entities because they play a vital role in how we define our table schema when adopting the Single Table design strategy. Also contained in this part 2 is Lambda function configuration with the Serverless Framework, configurations for creating DynamoDB table, keys and Global Secondary Index.
Entities
The Tenant Service contains these entities tenant, payment and property. Let's take a look at the Tenant entity.
import {v4} from "uuid";
export enum TenantStatus {
Active = "Active",
InActive = "Inactive"
}
export class Tenant {
readonly id: string;
readonly PK: string;
readonly SK: string;
readonly name: string;
readonly phone: string;
readonly status = TenantStatus.InActive;
readonly Type: string = Tenant.name;
readonly GSI1PK: string;
readonly GSI2PK: string;
constructor(data: Partial<Tenant>){
this.id = data.id ?? v4();
this.name = data.name;
this.phone = data.phone;
const { PK, SK } = Tenant.BuildPK(this.id);
this.PK = PK;
this.SK = SK;
const { GSI1PK, GSI2PK } = Tenant.BuildGSIKeys({ id: this.id });
this.GSI1PK = GSI1PK;
this.GSI2PK = GSI2PK;
}
static BuildPK(id: string) {
return {
PK: `tenant#id=${id}`,
SK: `profile#id=${id}`
}
}
static BuildGSIKeys(prop?: {id: string}) {
return {
GSI1PK: `tenant#id=${prop?.id}`,
GSI2PK: `type#${Tenant.name}`
};
}
}
In the src/tenant.ts file, we have pretty self-explanatory fields and two static functions. We also have an enumeration that is used to categorise tenants into;
The static functions, BuildPK and BuildGSIKeys are for formatting primary keys for both our default table and for Global Secondary Index (GSIs). These keys help with data manipulation in DynamoDB. These static functions are used as a convention in the Tenant Service such that the BuildPK is used to format the Primary key and BuildGSIKeys is used to format GSI keys where applicable. More about keys in the DynamoDB section.
Lambda Configuration
Lambda function configuration refers to parameters needed for the execution of a lambda function. This includes the function trigger and memory requirements, environment variables, AWS role and other services that the function will need to interact with. The Serverless Framework makes developing and deploying a Lambda function easy. However, it is up to you to organise your code in your preferred way that will be easy to navigate. I have organised the Tenant service in such a way that each lambda function resides in a separate directory to isolate them from the other functions. This provides a clear visual indication that all the required files for a function are located within a single directory.
The ./lambda directory contains all the functions that make our API. I prefer to name a function directory using verbs as it is easy to spot where a function resides based on its name and the action it performs. Let's take a look at the CreateTenant function.
CreateTenant Function
The CreateTenant directory contains three (3) files; config.yml, schema.ts and index.ts.
Config.yml
The config.yml file contains necessary function configurations, description, deployment and invocation mechanism using the Serverless Framework. Every necessary configuration as per application requirement for this function has to be declared here. The important keys in this file are described below:
handler - points to the TypeScript function to be invoked when the corresponding event is triggered - in this case, it is the main function.
memorySize and timeOut - has to do with the amount of memory needed and execution duration of the function
environment: key for specifying environment variables if needed
iamRoleStatements: lists permissions and the object of action in the AWS ecosystem. In the configuration below we have the dynamodb:PutItem action with permission set to allow on a database table resource via its resource name.
events: lists the triggers for the function - in this case, it is http which refers to an invocation from API Gateway. It also specifies the request method as POST and its path as /tenants.
CreateTenant:
handler: ./lambda/CreateTenant/index.main
description: "Create new tenant"
memorySize: 512
timeout: 10
environment:
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:PutItem
Resource:
- ${self:custom.DatabaseTable.arn}
events:
- http:
method: POST
path: /tenants
schema.ts
This file contains the JSON schema for validating the request payload for the CreateProperty function. This JSON schema is used by middy middleware to validate the request payload before the lambda function is executed. An error is thrown if the shape of the payload does not conform to the schema.
export default {
type: 'object',
required: ['body'],
properties: {
body: {
type: 'object',
required: ['name', 'phone'],
properties: {
name: {type: 'string'},
phone: {type: 'string'}
}
}
}
} as const
index.ts
contains the code that will be executed when the function is triggered
handler is a function of type ValidatedEventAPIGatewayProxyEvent<T>. It receives an argument name event that has been formatted to match a specified schema via the generic <typeof S> construct. The event argument contains event-related fields which include body, queryStringParameters and pathParameters fields from where the user data is gotten.
The exported main function is lambda's entry point. It takes in the handler function and schema field via the middyfy function and chains jsonBodyParser, validator and httpErrorHandler middlewares together before finally calling the handler function.
const handler: ValidatedEventAPIGatewayProxyEvent<typeof schema> = async (event) => {
const { phone, name }: { [key: string]: any } = event.body;
const tenant = new Tenant({ name, phone });
try {
await ddbDocClient.send(new PutCommand({
Item: tenant,
ConditionExpression: 'attribute_not_exists(phone)',
TableName: process.env.TENANT_TABLE_NAME
}));
return formatJSONResponse({
message: `Tenant created`,
tenant
}, 201);
}catch (e) {
const { message, statusCode } = new InternalServerError(`Create tenant failed`);
return formatJSONResponse({ e, message }, statusCode);
}
};
export const main = middyfy(handler, schema);
Type definition function
The ValidatedAPIGatewayProxyEvent is the type definition for the handler function used in index.ts. This function sets the shape of the body and queryStringParameters fields in the request payload proxied by API Gateway.
This src/libs/api-gateway.ts file also contains a function formatJSONResponse that formats response for API Gateway.
import type { APIGatewayProxyEvent, APIGatewayProxyResult, Handler } from "aws-lambda"
import type { FromSchema } from "json-schema-to-ts";
type ValidatedAPIGatewayProxyEvent<S> = Omit<APIGatewayProxyEvent, 'body' | 'queryStringParameters'>
& { body: FromSchema<S> }
& { queryStringParameters: FromSchema<S> }
export type ValidatedEventAPIGatewayProxyEvent<S> = Handler<ValidatedAPIGatewayProxyEvent<S>, APIGatewayProxyResult>
export const formatJSONResponse = (response: Record<string, unknown>, statusCode = 200) => {
return {
statusCode,
body: JSON.stringify(response)
}
}
DynamoDB Configuration
Single-Table approach...certainly not for the faint-hearted.
- Paul Swali
Designing for a relational database software system sometimes requires that one achieves normalization - a 3NF, by splitting data into separate tables where each table represents an entity. You could have a join table as needed. The Single Table design requires a different approach which could be challenging at least for the first time and in some complex scenarios.
It is important to note that DynamoDB does not constrain you to the use of only the Single table design. You are welcome to design your DynamoDB tables as though you were working on a relational database system with several tables in place. With that said, let's dive into the Tenant service table schema.
The configuration for the DynamoDB resource is found in ./resource/database-table.yml which contains some sort of DDL statement. Given the fact that all entities will be stored in a single table, we have to define generic keys that will be meaningful across all entities. In other words, the attributes selected to serve as keys and ensure data integrity must apply to Tenant, Payment and Property entities. To make that generic and easy to manipulate, it is best to name them by their function. For the Tenant service, PK refers to the partition key and SK, sort key.
The DynamoDB resource file database-table.yml reveals self-descriptive fields except for a few. The table is designed to have a composite primary key and two Global Secondary Indexes that have their respective primary keys; GSI1PK and GSI2PK. The table also enables DynamoDB Streaming via the StreamSpecification key.
TenantServiceTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:service}-${self:provider.stage}-Table
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: PK
AttributeType: S
- AttributeName: SK
AttributeType: S
- AttributeName: GSI1PK
AttributeType: S
- AttributeName: GSI2PK
AttributeType: S
KeySchema:
- AttributeName: PK
KeyType: HASH
- AttributeName: SK
KeyType: RANGE
StreamSpecification:
StreamViewType: NEW_IMAGE
GlobalSecondaryIndexes:
- IndexName: GSI1
KeySchema:
- AttributeName: GSI1PK
KeyType: HASH
Projection:
ProjectionType: ALL
- IndexName: GSI2
KeySchema:
- AttributeName: GSI2PK
KeyType: HASH
Projection:
ProjectionType: ALL
DynamoDB Keys - PK & SK
A composite primary key is a primary key that consists of more than one attribute - which is used to uniquely identify a record in a table. Leveraging the composite key concept, we have the PK and SK defined in the KeySchema field. The primary key determines the Data Access Patterns in DynamoDB which is how data is accessed within a table. Care has to be taken when deciding on how a primary key is constructed. In the snippet below, we see how the Tenant's entity primary key is constructed; an id is passed in as an argument to the BuildPK function and we have an object with two attributes as the primary key - this primary key has to match the requirement of the primary key definition in the ./resource/database-table.yml file.
static BuildPK(id: string) {
return {
PK: `tenant#id=${id}`
SK: `profile#id=${id}`
}
}
To retrieve a tenant's record from the table, this primary key is used. By using the primary key, a full table scan is avoided because of its performance and cost implications. We may need to retrieve a tenant's record by a different attribute other than the initial primary key. When this happens we then need to create an alternative primary key typically seen as another Data Access Pattern. In doing so, a tenant record can be retrieved based on that attribute that now forms the primary key. How can this alternative primary key be created?
Given the needs of an application, multiple Data Access Patterns would need to be utilized for effective data retrieval
Global Secondary Index
To keep it simple, a Global Secondary Index - (GSI) enables you to create an alternative primary key using a different attribute. This ultimately provides more options with which we can query a DynamoDB table.
The Tenant entity has two fields; GSI1PK and GSI2PK that serve as primary keys to the GSIs defined in the GlobalSecondaryIndexes key. There are two indexes; GSI1 and GSI2, the key type and projection which specifies the attributes you want to return when that index is used.
The Payment and Property entities have just one GSI key which is the GSI1PK . In Part 3 of this series, we will look at the usefulness of having a generic key such that entities share similar keys but with different functionality. It is important to note that GSIs and their corresponding keys are added on a per-need basis - the needed Data Access Pattern should determine what keys are created and their composition.
Keys In Action
const handler: ValidatedEventAPIGatewayProxyEvent<typeof schema> = async (event) => {
const { status }: {[key: string]: any} = event.queryStringParameters;
const { GSI2PK } = Tenant.BuildGSIKeys();
const result = await ddbDocClient.send(new QueryCommand({
TableName: process.env.TENANT_TABLE_NAME,
IndexName: GSIs.GSI2,
KeyConditionExpression: 'GSI2PK = :gsi2pk',
FilterExpression: '#status = :status',
ExpressionAttributeValues: {
':gsi2pk': GSI2PK,
':status': status,
},
ExpressionAttributeNames: {
'#name': 'name',
'#status': 'status'
},
ProjectionExpression: 'id, #name'
}));
const tenants = result.Items.map(item => item as Pick<Tenant, 'id' | 'name'>);
return formatJSONResponse({
count: tenants.length,
tenants
});
}
The snippet above contains a DynamoDB Query command operation where an Index and its corresponding primary key are used to fetch one or more records from a DynamoDB table. This is achieved using the IndexName and KeyConditionExpression keys. You can also specify filter conditions using the FilterExpression key as necessary.
Conclusion
In this article we took a step further into entities and their composition, using the tenant entity as a case study. We also looked at keys, how they were constructed and their usefulness in index operations.
Lambda configuration came next as we looked at how the Serverless Framework applies our configuration for AWS Lambda. The configuration also affects how functions are written as we have to specify the services, interactions and the actual operations (permissions) that can be carried out by the function.
A major section of this article looked at the DynamoDB table service. While this isn't an in-depth review of the DynamoDB service and features, we saw the usefulness of primary key - composite and non-composite types, Global Secondary Index and how you can select what Index you intend to use for a query operation.
In Part 3 of this article, we will go deeper into how primary keys determine what data is returned. We will also see DynamoDB streams in action - with this, you can track changes to your table immediately after a DML operation without issuing another query.
