Jul 12, 2022

Use AWS SNS to send data to Tinybird

SNS is a popular pub/sub messaging system for AWS users. Here's how to use SNS to send data to Tinybird.
David Manzanares
Software Engineer

If you operate on AWS infrastructure, AWS’s Simple Notification Service (SNS) provides a simple managed service for pub/sub messaging to various AWS and external services.

Given that a lot of Tinybird customers operate on AWS, SNS tends to be one of our most requested data origins. It provides an easy-to-use path to distribute messages to multiple destinations, allows for message filtering, and has built in data-loss prevention mechanisms. Because it’s a tool our customers regularly use and like, we’ve prioritized making it possible to connect SNS to Tinybird.

And now, we are happy to say that we can support data ingestion from SNS! We’ve conditionally rolled it out for all customers and are opening up the feature to feedback. If you have any, please share it with us in our community Slack.

How to use SNS to send data to Tinybird

Here’s how to use AWS SNS to send data to Tinybird:

Step 1: Use an existing SNS topic or create a new one

From your AWS console, navigate to an existing SNS topic or create a new one.

A webpage showing how to create an SNS topic to send data to Tinybird
Create an SNS topic to send data to Tinybird.

Step 2: Create a new subscription for the topic

Define a new subscription using the Tinybird /events API endpoint as the subscription Endpoint using the following syntax:

https://api.tinybird.co/v0/events?name=YOUR_DATA_SOURCE_NAME&token=YOUR_TOKEN

Make sure to select HTTPS as the protocol and check “Enable Raw Message Delivery”, then click “Create Subscription.”

A webpage showing how to create a subscription from AWS SNS to the Tinybird /events endpoint
Creating a subscription to an SNS topic using the Tinybird /events endpoint.

You should see a confirmation screen like the one below.

AWS SNS confirmation screen
Success!

Step 3. Start sending messages to your SNS topic

You can manually publish messages to your new SNS topic from the topic page by clicking “Publish Message”

A webpage showing how to publish a message to an SNS topic.
Publish a message to your SNS topic to test the connection.

Just define the JSON payload to be sent in the Message Body section

A webpage to publish a message to an SNS topic showing the JSON message structure
Use JSON structure to send data to /events.

You should see data appear in your Tinybird data source.


A Tinybird data source
Boom. Data in Tinybird!

Here’s a bit of JavaScript showing how to publish a new message to the SNS topic. Just replace your AWS region and Topic ARN in the code.

As always, if you have any questions or run into issues, you can join our Slack community.

Do you like this post?

Related posts

Tinybird has joined the AWS ISV Accelerate Program
Tinybird
Team
Jun 05, 2024
Tinybird expands self-service real-time analytics to AWS
We've improved notifications for ingestion issues
Run analytics on files in Amazon S3 the easy way
Tinybird
Team
Jul 20, 2023
Using custom Kafka headers for advanced message processing
How to set up event-based ingestion of files in S3 for free
Tinybird connects with Confluent for real-time streaming analytics at scale
Tinybird
Team
Jul 18, 2023
Export data from Tinybird to Amazon S3 with the S3 Sink
Tinybird
Team
Mar 21, 2024
Simplifying event sourcing with scheduled data snapshots in Tinybird
Introducing the Tinybird DynamoDB Connector in public beta

Build fast data products, faster.

Try Tinybird and bring your data sources together and enable engineers to build with data in minutes. No credit card required, free to get started.
Need more? Contact sales for Enterprise support.