Role-based AWS Setup for Mulesoft S3 Connector's On New Object Component (Including Local Run Setup)
This document covers setting up AWS Services when using Mulesoft S3 Connector's On New Object component. We'll cover the minimum permissions required (Least-Privilege Access) by this component. The S3 Connector here is configured to use AWS IAM Role-based authentication which is only applicable for CloudHub 2.0 deployments targeted inside a Private Space. The role-based authentication is only supported by latest versions of this connector viz. 6.2.0+. The On New Object component is an event source which triggers Mule flow whenever a new file is uploaded in a S3 Bucket. It sends metadata of the uploaded file as payload. For this component to work, the S3 Bucket must have a notification configuration attached to it with destination set to an SQS Queue.
In the end, we'll look over how to deploy and run such Mulesoft Application locally.
Let’s begin...
Pre-requisites
- Anypoint Studio (S3 Connector V6.2.0+)
- Mulesoft Anypoint Platform Account
- Private Space in Anypoint Platform
- AWS Account
- S3 Bucket and SQS Queue in AWS
Anypoint Platform Setup
First, let's enable AWS Service Role in your Private Space. Login to your Anypoint Platform Account and head over to the Runtime Manager. From the left-hand side menu, click on Private Spaces. Then click on the Private Space that you have created.
.jpg "Anypoint Platform: Private Space")
Here, click on the Advanced tab and scroll to the bottom. Under the AWS Service Role section, check ✔ mark the Enable AWS Service Role box and click Save Changes.
.jpg "Anypoint Platform: Private Space | Enable Service Role")
Copy the Role ARN generated by Mulesoft. We'll use it when setting up Trust policy in AWS Role.
SQS Queue Access Policy
Let's setup the SQS Queue Access Policy first. Login to your AWS Account and head over to the SQS Queue you have. Click on the Access Policy tab and then click on the Edit button.
Now, modify the policy as shown below and Save changes.
Policy Snippet -
S3 Bucket Notification Configuration
Now, let's setup the S3 Bucket Notification Configuration. Open the Properties section of the S3 Bucket you wish to use.
Scroll down to the Event notifications section and click on Create event notification button.
Type in the Event name of your choice (This will be used in On New Object component in Anypoint Studio later). Also, in the Event types section, check ✔ mark the All object create events box.
Scroll down further to the Destination section and here select the SQS queue you just configured from the drop-down. Click Save changes.
Please note that if you have not configured the Access Policy of your SQS Queue before this step, then AWS won't allow you to Save changes here.
IAM Role Setup
Now, we create an IAM Role. In the IAM dashboard, from the left-hand side menu, click on Role and then click on Create role button.
Now, in the Step 1, select Custom trust policy and modify the policy as shown below.
Policy Snippet -
Click Next.
Now, in the Step 2, click on Create policy button.
A new browser tab will open. Here, you'll be creating a new policy. Switch to the JSON format and modify the policy as shown below.
Policy Snippet -
Please note that the Statement3 here grants Full Read Write access to the contents of S3 Bucket and you can modify/remove this as per you requirements.
Click Next. Give this policy a Name and then click Create policy.
Now, go back to the previous tab. Click on the Refresh button and then find the policy you just created using the search field. Check ✔ mark the box in front of the policy to attach it to your role and click Next.
Give this role a Name and then click Create role.
Now, the IAM Role is created, let's copy its ARN as we will use it in Anypoint Studio.
S3 Connector Configuration
In your Anypoint Studio, drag and drop the On New Object component from S3 Connector (V6.2.0+). In the Connector configuration, set Try Default AWSCredentials Provider Chain to True. Choose Edit inline option in Role drop-down and in the ARN field, paste the ARN of IAM Role that you just created in previous step.
.jpg "S3 Connector Configuration: Role ARN")
Scroll down and set a dummy value in Access Key and Secret Key fields. Also, check the AWS Region Endpoint field. It must be the same region where your S3 Bucket and SQS Queue lives.
.jpg "S3 Connector Configuration: Keys & Region")
In the On New Object component's General section, enter your S3 Bucket name as highlighted below.
.jpg "S3 On New Object: General")
Also, under the Advanced section, in Configuration Name field, enter your S3 Bucket's Event Notification Configuration Name.
.jpg "S3 On New Object: Advanced")
Please note that, if you try to run the code in local environment it will not work. The role-based authentication will only work for the code deployed in CloudHub 2.0 and Private Space.
Local Run Setup
As previously mentioned, the role-based authentication won't work when code is deployed locally. To run this code in local environment, we need an IAM user which can assume an IAM role. For authentication, we will need to generate Access keys in IAM user configuration and for authorization, we will attach a permission policy to the IAM role.
So, let's create the IAM user as shown -
There is no need to add this user to any User groups or attach any permissions to it. Click Next.
Review and click Next.
Copy the IAM user ARN as we'll need it when we create the IAM role. Scroll down to the Access keys section and click Create access key button.
Select Other in the Use case section and click Next. Add any description tag if you want and then click Create access key button to finish creating access keys for this IAM user.
Copy or Download the Access key and Secret access key as we'll need them later. Click Done.
Next, we create another IAM role and setup its Trust policy such that our above created IAM user can assume it.
Policy Snippet -
Click Next. On the Add permissions step, attach the same Permission policy here that we attached to our previously created IAM role and click Next.
Give this IAM role a Name and click Create role.
Once created, copy the IAM role ARN and save it somewhere.
Finally, open the S3 Connector Configuration in Anypoint Studio. Turn the following fields value into Property Placeholders (Link) -
- Try Default AWSCredentials Provider Chain (Switch to Configuration XML mode to turn this field into a property)
- Role ARN
- Access Key
- Secret Key
Now, whenever you run the code in local environment, just use Runtime variables to override those properties with the following values -
- Try Default AWSCredentials Provider Chain = False
- Role ARN = IAM role ARN that we just created
- Access Key = Access Key of the IAM user we just created
- Secret Key = Secret Access Key of the IAM user we just created
Remember that for CloudHub deployments, the values of those Runtime variables will be different as explained in previous sections.
That's it.
Comments
Post a Comment