Building a CDK Construct Library: thoughts and tips

I recently created a CDK library provisioning a CI/CD pipeline for serverless apps on AWS. Having finished the “regular” JS library, it took me a day to convert it to a “proper” CDK Construct library. Not long, and I learned a lot. Here are my thoughts and tips.

What do I mean by a “proper” CDK Construct library? One that:

• can be automatically published for multiple languages (with only one source code!),
• and is listed on the Construct Hub.

To achieve this, you need to use the jsii library, as does the CDK itself. But before jumping straight to building CDK Construct libraries with jsii, we must take a step back.

How CDK Really Works

To understand building CDK libraries, first, we need to learn how CDK works.

CDK is available in 5 programming languages: TypeScript/JavaScript, Python, Java, C#, and Go. In all of them, it offers the same Constructs and capabilities.

As you can imagine, developing a library of the size of CDK, with over 600k lines of code at the time of writing, is not easy. Doing so for 5 languages increases the effort tremendously, adding the complexity of synchronization, inevitable discrepancies between implementations, and language-specific bugs.

That’s why AWS CDK is written only in TypeScript. Then, the jsii library generates code in other supported languages.

But not all CDK code is converted to other languages. Instead, the jsii generates only public interfaces in other languages and bindings to execute the original JavaScript code. So when you run CDK code in, let’s say, Python, under the hood CDK invokes JavaScript.

Why is that?

In short: programming languages offer different capabilities, and transpiling the code between them is challenging, to say the least. While such transpilers exist for specific cases, no tool can convert the code between 2 arbitrary languages and produce optimal output. Not even the AI, which will supposedly make developers obsolete 🙃

The ultimate advantage of this solution is that CDK is available in 5 languages, with a concise set of features, and all changes are immediately released for all platforms. The disadvantage is suboptimal performance. However, that’s a trade-off acceptable for the build tools.

Now, one of the main powers of CDK are reusable custom Constructs. What if you want to share them? Can you use the same approach as CDK – to write code once and have it available in all supported languages?

Yes. And it’s easier than you may suspect.

Building CDK Construct libraries with jsii

As I mentioned, the tool that takes your TypeScript sources and builds CDK Construct libraries in multiple languages is jsii. Using it comes with possible pitfalls and limitations I cover below. However, it should be straightforward to integrate.

As the official docs and several articles already covered the basics of using the jsii to build CDK libraries, let’s focus on more interesting, practical bits.

The Good

TypeScript 5 (and beyond)

This was supposed to be the main “bad parts” point. Until recently, jsii supported only TypeScript 3.9, released in May 2020. And a lot of good has been added to TypeScript since then.

But it’s no longer the case. From now on, new jsii versions follow TypeScript releases. That means you can use the full capabilities of the latest TypeScript versions while building CDK libraries.

Packaging and publishing automation

If you have ever created a JavaScript library, you know it could be simpler. Should you build it as CommonJS, ES Modules, or both? How to set up package.json parameters so it works for everyone? And provide TypeScript types?

And that’s just JavaScript, while we want to build for 4 more languages. Agreed, probably the JavaScript ecosystem is the most complicated. But building proper packages for other platforms is still challenging, especially if you don’t know them well.

Fortunately, all of this is handled by the jsii-pacmak tool. No decisions to make on your side. jsii-pacmak will produce packages for all 5 languages based on the common configuration and a few language-specific properties (like Python module name or Java package name).

And, no matter how amusing it sounds, if you don’t want to implement publishing packages to 5 different registries yourself, you will also like the publib tool. More on that in The Tools section below.

Publishing in Construct Hub

You’ve created a CDK library and now want people to see it? The go-to place to discover new CDK packages is the Construct Hub.

The good news is you don’t need to do anything extra to get your library listed there. It will be automatically added there if:

• it’s published in the npm package registry,
• under open source license,
• and includes the awscdk keyword in the package.json.

The Bad

Public interface restrictions

In CDK, you create Constructs customized with properties. In TypeScript/JavaScript, you pass them as an object:

new s3.Bucket(this, 'MyBucket', {
  versioned: true,
});

It’s similar in other languages. In Python, you pass keyword arguments instead of an object:

s3.Bucket(self, 'MyBucket', versioned=true)

And in Java, you use the builder pattern:

Bucket.Builder.create(this, "MyBucket").versioned(true).build();

While writing custom Constructs, you define those properties as an interface:

export interface BucketProps {
	readonly versioned: boolean;
}

When creating a CDK Construct library, those interfaces define your public API and will be transpiled to other languages.

However, those languages differ in capabilities. While TypeScript/JavaScript is rather flexible, not all its semantics can be converted to Python, Java, C#, or Go. Therefore, jsii imposes some restrictions. For example, you can’t create public interfaces with generic types:

// not allowed with jsii
export interface MyProps<T> {
	readonly value: T;
}

I won’t list all the limitations here; you can review them in the documentation.

But to be clear: those restrictions only affect public interfaces – the API of your library. Nothing prevents you from creating internally used generic interfaces because, as explained earlier, internal code is left “as is” and not transpiled to other languages.

The Pitfalls

Bundling Lambda functions

If your CDK library contains a Lambda function, you should build it separately ahead of time and use regular lambda.Function instead of NodejsFunction.

Why? Above all, building library functions on each deployment is a waste of time since they do not change. But there are also multiple cases where it will simply fail. NodejsFunction requires either esbuild installed or Docker running. If you use any dependencies from your library root package.json, they will not be available unless you bundle them with your library. And finally, bundling may fail in CDK projects using a different language than JS/TS.

The same applies to the PythonFunction and similar.

The solution is to bundle Lambda functions code ourselves while building the library. Here is a little how-to.

Let’s say you have the following directory structure:

my-cdk-lib/
├── src/
│   ├── lambda/
│   │   ├── functionOne/
│   │   │   └── index.ts
│   │   └── functionTwo/
│   │       └── index.ts
│   ├── index.ts
│   └── myLibConstruct.ts
└── package.json

Start with setting some options in the jsii configuration to:

• use src as a source,
• output transpiled sources under lib,
• output built packages under dist,
• ignore src/lambda sources – they will be built separately.

{
	"jsii": {
		"outdir": "dist",
		"tsc": {
			"rootDir": "src",
			"outDir": "lib"
		},
		"excludeTypescript": [
			"src/lambda/**/*"
		]
	}
}

Now, add esbuild to the project and create an esbuild.mjs file:

import * as fs from 'fs';
import * as esbuild from 'esbuild';

const entryPoints = fs.readdirSync('src/lambda', {withFileTypes: true})
    .filter(dirent => dirent.isDirectory())
    .map(dirent => `src/lambda/${dirent.name}/index.ts`)
    .filter(file => fs.existsSync(file));

await esbuild.build({
    entryPoints,
    bundle: true,
    platform: 'node',
    outdir: 'lib/lambda',
});

In CDK, point to the function sources as usual:

new lambda.Function(this, 'MyFunctionOne', {
	code: Code.fromAsset(path.join(__dirname, 'lambda', 'functionOne')),
});

Finally, to build the library, run:

node esbuild.mjs && jsii

This will output both the CDK Constructs and Lambda functions code to the lib directory, from which later you can create distribution packages with jsii-pacmak.

To reduce the library package size, it may be a good option to use AWS SDK provided in the Lambda runtime and Lambda Layers with libraries like Lambda Powertools. In such a case, you can pass a list of external dependencies to not bundle in functions code with the esbuild external option.

A similar approach can be used if your library contains a frontend application.

Using dependencies

CDK libraries don’t fit into traditional dependency management patterns. You can’t declare a package in package.json dependencies section to be installed together with your library.

Why? Because your library can (at least theoretically) be installed in Python, Java, C#, or Go project, using a package manager completely oblivious of your Node.js dependencies.

Does it mean you can’t use dependencies at all? Fortunately, no.

The first solution is to use other jsii libraries. You can safely declare them as dependencies as long as they are published for the same runtimes as your library. But this applies only to the libraries reusing other CDK libraries. What about “regular” ones?

You can declare other libraries in bundleDependencies section of the package.json file. Those dependencies will be, well, bundled together with your library. Meaning – they will be copied from node_modules and added to the zip file with your library. Of course, you should use this only in necessity – it can quickly bloat your library package size. Thus, it’s best to avoid using external libraries in your CDK library code altogether.

The Tools

There are several helpful tools around the jsii.

publib

There are 3 steps to create a CDK library:

1. Build – transpile from TS to JS
2. Package – create Node.js, Python, Java, C#, and Go packages
3. Publish – send them to appropriate package registries (npmjs.com, PyPI, etc.)

The first two steps are handled by the jsii and jsii-pacmak tools.

publib is a handy utility to ease the last part. Given credentials to appropriate registries, it publishes packages to each of them. Moreover, it’s preconfigured to work just fine with the output of jsii-pacmak.

aws-delivlib

If you want to automate building and publishing CDK packages, aws-delivlib is a CDK Construct that provisions a CI pipeline on AWS specifically for this. It’s used by the CDK itself.

projen template

I’m still not convinced about the projen, but it’s used by many CDK libraries. So there is a projen template for CDK Construct Library to help with that.

Conclusion

Building CDK Construct libraries isn’t hard. If you understand how CDK works, most things make sense. Just remember:

When in a Python CDK app you use a Construct library containing a Lambda function bundled with esbuild…

Python invokes Node.js to deploy code built with Go

What a time to be alive, right?

✖

Thank you for reading :-)

Join Newsletter and Stay Updated!

Join 460+ subscribers and get:

• 📨 algorithm-independent information about new posts
• 📘 my ebook about Serverless on AWS
• ✅ zero spam

First name

Email

Subscribe

Thank you! Check your email to confirm subscription.

You are already subscribed. Check your spam folder for newsletter emails.

I'm sorry, something went wrong. Please try again later.

One email every few weeks.
Unsubscribe anytime.

✖

Before you leave...

Get Serverless Ebook!

Join the newsletter and get my free ebook about → serverless best practices, → Infrastructure as Code, → AWS services, and → architecture patterns.

First name

Email

Subscribe

Thank you! Check your email to confirm subscription.

You are already subscribed. Check your spam folder for newsletter emails.

I'm sorry, something went wrong. Please try again later.

One email with a new post every few weeks.
Unsubscribe anytime.