Amazon Web Services (AWS): Boto3 SQS Operations

Amazon Web Services (AWS): Boto3 SQS Operations

Boto3 Vineet Sachdeva   February 08, 2020

In this blog I am going to cover on how to run multiple SQS Operations using Boto3. 
Note - all the response from which are printed will give HTTP Status Code 200 which signifies that the operation which you had performed had completed successfully.
Create Queue

import boto3

QName = "MyNewFIfoQueue.fifo"Attr = {'FifoQueue': 'true'}
sqs = boto3.resource('sqs')
response = sqs.create_queue (QueueName = QName, Attributes = Attr)
print (response)import boto3

QName = "MyNewFIfoQueue.fifo"Attr = {'FifoQueue': 'true'}
sqs = boto3.resource('sqs')
response = sqs.create_queue (QueueName = QName, Attributes = Attr)
print (response)

And the response 

sqs.Queue(url='https://queue.amazonaws.com/<accid>/MyNewFIfoQueue.fifo')

You can verify the same from Console or AWS CLI

$ aws sqs list-queues
{
    "QueueUrls": [
        "https://queue.amazonaws.com/<accid>/MyNewFIfoQueue.fifo"    ]
}

Send Messages to Queue

import boto3
import uuid

QName = "MyNewFIfoQueue.fifo"
sqs = boto3.client('sqs')
QUEUE_URL = sqs.get_queue_url(QueueName = QName)['QueueUrl']
response = sqs.send_message(
                           QueueUrl=QUEUE_URL,
                           MessageBody="TestMessage",
                           MessageGroupId="TestGroup",
                           MessageDeduplicationId=str(uuid.uuid4()))
print(response)


And the response
{u'MD5OfMessageBody': '08ff08d3b2981eb6c611a385ffa4f865', 'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': 'b18d5e81-a666-541b-aaa4-0839850e0a6e', 'HTTPHeaders': {'x-amzn-requestid': 'b18d5e81-a666-541b-aaa4-0839850e0a6e', 'date': 'Sat, 08 Feb 2020 02:47:04 GMT', 'content-length': '431', 'content-type': 'text/xml'}}, u'SequenceNumber': '18851513359964655616', u'MessageId': '3a9b64ad-4eb3-4606-9eec-c9c5369d4304'}






Receive messages From Queue




import boto3
QName = "MyNewFIfoQueue.fifo"
sqs = boto3.client('sqs')
QUEUE_URL = sqs.get_queue_url(QueueName = QName)['QueueUrl']
response = sqs.receive_message(
                           QueueUrl=QUEUE_URL,
                           MaxNumberOfMessages=1                           )['Messages'][0]
print(response)


And the response 
{u'Body': 'TestMessage', u'ReceiptHandle': 'AQEBmnx4wSlS0Qf/kramgeBUr8lMEHrWkILeK3SIoxMjfnMjRGrXtm8w8BUXiiKJQSaFYaGYnJF6kpFrYeFPoGlrVcJgn6Ci3WpM+pVm1Ih0XT4SkHQBjH2CIxKfx21t+oyej7mYi3PwNENOHJI125BNuAVnfSAys64uBFPXgEPgRy/OFBVK2CcueJy18I8sPm6dNV5CCzxfzZE3csd/TBOQsnhtAPt3sro3MfZUUUc5d3iIrhGjVa/xNXiNNHECMu5ZifCTU8U1pX2lX1EwV3CYzrlnr2mie/R6SkJqEvPjsfc=', u'MD5OfBody': '08ff08d3b2981eb6c611a385ffa4f865', u'MessageId': '3a9b64ad-4eb3-4606-9eec-c9c5369d4304'}






Purge and Delete Queue

import boto3

QName = "MyNewFIfoQueue.fifo"
sqs = boto3.client('sqs')
QUEUE_URL = sqs.get_queue_url(QueueName = QName)['QueueUrl']
response = sqs.purge_queue(QueueUrl=QUEUE_URL)
print(response)
response = sqs.delete_queue(QueueUrl=QUEUE_URL)
print(response)

And the response

{'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '364d9e6a-9b1b-555e-b5af-c8d2bd4abe6a', 'HTTPHeaders': {'x-amzn-requestid': '364d9e6a-9b1b-555e-b5af-c8d2bd4abe6a', 'date': 'Sat, 08 Feb 2020 02:59:29 GMT', 'content-length': '209', 'content-type': 'text/xml'}}}
{'ResponseMetadata': {'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '75808c73-aac5-5dec-b3a4-dacb8c9446d8', 'HTTPHeaders': {'x-amzn-requestid': '75808c73-aac5-5dec-b3a4-dacb8c9446d8', 'date': 'Sat, 08 Feb 2020 02:59:29 GMT', 'content-length': '211', 'content-type': 'text/xml'}}}

Amazon Web Services (AWS): Enabling API Access

Amazon Web Services (AWS): Enabling API Access

Boto3 Vineet Sachdeva   January 30, 2020

The First step of enabling API access is to get your AWS Access keys properly created and defined. 

Step 1 - Login to AWS console and Navigate to IAM

Step 2 - Navigate to users and select user which you want to have the access key 

Step 3 - Go to Security Credentials and Click on Create Access Key. 

Remember, you will get a pop-up and this is the only time you will able to see the Secret Access Key, so it is a good choice to download and save it somewhere and probably somewhere safe as it gives access to your account. 
However, if you loose it, contact your admin and the admin can delete it or make it inactive

Okay, Now that you have 2 parts of Key
Access Key and Secret Access Key.

You should now configure your environment
In your home directory create a folder named .aws and create the credentials file as below.

cat ~/.aws/credentials
[default]
aws_access_key_id=xxxxxxxx
aws_secret_access_key=xxxxx

Now this will be used as your default credentials.

You should also create a config file which specifies default region 
Example Below 
cat ~.aws/config
[default]

region=us-east-1

The other option can be to set it as your environment variables or set it as your environment variables in the Code Editor you use. 
I use pycharm, so go to Run and Edit Configurations, you will see environment  variables there which you can configure. 

Also, you can use boto3 quick start for your help as well.

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html

AWS : Lambda: Add IP to Security Group Using Boto3 - Complete Code

AWS : Lambda: Add IP to Security Group Using Boto3 - Complete Code

Boto3 Vineet Sachdeva   January 07, 2020

IN this blog I am going to show how to add an IP using Lambda.

You can create a sample-SQS trigger event with an IP address in body to create an SQS event emulation for testing..

Now, 

lamda_handler is the default handler for lambda
It checks if the IP is not already part of the rule
it then calls updateIP and refreshes the timestamp if yes or adds new with new timestamp if no
the revoke is to temporarily revoke and add the IP.

import boto3
from datetime import datetime

ec2 = boto3.resource('ec2')
s_group = ec2.SecurityGroup('sg-85d42ac2')
dt = datetime.now()
date_format = "%m-%d-%Y %H:%M"str_dt = dt.strftime(date_format)
ssh_port = 22code = 200max_minutes = 5

def lambda_handler(event, context):
    for record in event['Records']:
        ip = record["body"]
        if (str(ip) == 'sweep'):
            sweepIP()
        else:
            verifyAddIP(str(ip))

def verifyAddIP(strIP):
    m_strIP = strIP + '/32'
    ip_permission = s_group.ip_permissions[0]
    ip_range = ip_permission['IpRanges']

    for cidr in ip_range:
        if (cidr['CidrIp'] == m_strIP):
            updateRule(strIP + '/32', True)
        else:
            updateRule(strIP + '/32', False)


def updateRule(strIP, update_p):
    if update_p:
        response = s_group.revoke_ingress(IpProtocol="tcp", CidrIp=strIP, FromPort=ssh_port, ToPort=ssh_port)
        response = s_group.authorize_ingress(IpPermissions=[
            {'IpProtocol': 'tcp',
             'FromPort': ssh_port,
             'ToPort': ssh_port,
             'IpRanges': [{'CidrIp': strIP, 'Description': str_dt}]
             }
        ]
        )
        print ('Update IP Address Time in Ingress Rule - ' + strIP)
    else:
        response = s_group.authorize_ingress(IpPermissions=[
            {'IpProtocol': 'tcp',
             'FromPort': ssh_port,
             'ToPort': ssh_port,
             'IpRanges': [{'CidrIp': strIP, 'Description': str_dt}]
             }
        ]
        )
        print ('Added IP Address to Ingress Rule - ' + strIP)

AWS : Lambda: Remove IP to Security Group Using Boto3

AWS : Lambda: Remove IP to Security Group Using Boto3

Lambda Vineet Sachdeva   December 05, 2019

In this blog I discuss on how to remove an IP from Security Group using Boto3

import boto3

ec2 = boto3.resource('ec2')

s_group = ec2.SecurityGroup('sg-<ID>')

response = s_group.revoke_ingress(IpProtocol="tcp", CidrIp=strIP, FromPort=22, ToPort=22)
print (response)

Here strIP : IP Range - Example 10.24.25.0/24
From Port and To Port are port Ranges 



With the above you can revoke an IP address rule

AWS: Boto3: Send Message Queue

AWS: Boto3: Send Message Queue

Cloud Vineet Sachdeva   December 05, 2019

In this simple example, I configure AWS Boto3 to send a message to known queue.

import boto3
sqs = boto3.resource('sqs')
queue = sqs.get_queue_by_name(QueueName='Your_Queue_Name')
response = queue.send_message(MessageBody='BodyText')
print(response.get('MD5OfMessageBody'))

Note - you must configure your Access Credentials, the way they can be done is given in the URL Below 

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#guide-configuration



I generally prefer to set it as my shell variable, but it is totally up to you.

The user/object with which you are accessing should have SQS Policy Attached so that it can write to the Queue.

AWS : Lambda: Add IP to Security Group Using Boto3

AWS : Lambda: Add IP to Security Group Using Boto3

Lambda Vineet Sachdeva   December 05, 2019

In this blog I am going to show example on adding an IP address to AWS security group using Boto3.

The way is simple, just create your own Lambda and add the below Code. 
You can have trigger of SQS and put an example IP in the Body . 


import json
import boto3

ec2 = boto3.resource('ec2')
s_group = ec2.SecurityGroup('sg-0308cd0e895d42ac2')
# This is your Security group unique ID


def lambda_handler(event, context):
    failed = False;
    
    try:
      print ("The value IS " + s_group.group_id)
      for record in event['Records']:
        ip = record["body"]
        print (str(ip))
        response = s_group.authorize_ingress(IpProtocol="tcp", CidrIp=str(ip),FromPort=80,ToPort=80)
    except Exception:
      logger.exception("Failed to Add IP")
      # Add your failure function 
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }


Sample Event to Use

{
  "Records": [
    {
      "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
      "receiptHandle": "MessageReceiptHandle",
      "body": "10.2.3.0/32",
      "attributes": {
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "1523232000000",
        "SenderId": "123456789012",
        "ApproximateFirstReceiveTimestamp": "1523232000001"
      },
      "messageAttributes": {},
      "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
      "eventSource": "aws:sqs",
      "eventSourceARN": "arn:aws:sqs:us-east-1:123456789012:MyQueue",
      "awsRegion": "us-east-1"
    }
  ]
}

Reference: https://docs.aws.amazon.com/code-samples/latest/catalog/python-ec2-create_security_group.py.html