Friday, December 08, 2017

Accessing database in AWS - Part 2 ( Refactoring your code )




I have refactored the code to as below.
As you can see, I have refactored getRegistration into two methods. The first method is generic so that tomorrow if you want to use GetLoans, we could use the generic method. And also the method "getRegistration" now uses the generic method "getValueFromDatabase".
I am using the scan method of AWS ( which is more intensive) than query because I have to search on a column which is NOT a partition key.
If it was a partition key, I could have used "query" instead of "scan".
I am using if (scanResult.getCount() != 0) break;   to return from the scan immediately as soon as I get one result.
Else it would scan the entire database and I would get a "Throughput exceeded violation/error" from AWS.

import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.model.*;
import com.myob.beepy.RestAssuredFramework.Data.Env.EnvironmentConfiguration;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.junit4.SpringRunner;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = {EnvironmentConfiguration.class})
@TestPropertySource(locations = "classpath:enviromentConfiguration.yml")
public class DatabaseAccess {

    private final AmazonDynamoDB client;
    @Autowired
    EnvironmentConfiguration environmentConfiguration;

    public DatabaseAccess() {

        client = AmazonDynamoDBClientBuilder.standard().withRegion(Regions.AP_SOUTHEAST_2).build();
    }

    public List> getRegistration(String registrationId) {
        String registrationsTableName = environmentConfiguration.getTablePrefix() + "BF_Applications";
        String columnName = "loan_id";
        return getValueFromDatabase(registrationsTableName, columnName, registrationId);
    }

    private List> getValueFromDatabase(String tableName, String columnName, String valueOfColumnToScan) {
        HashMap scanFilter = new HashMap<>();
        ScanResult scanResult;
        Condition condition = new Condition()
                .withComparisonOperator(ComparisonOperator.EQ.toString())
                .withAttributeValueList(new AttributeValue(valueOfColumnToScan));
        scanFilter.put(columnName, condition);
        Map lastKeyEvaluated = null;
        do {
            ScanRequest scanRequest = new ScanRequest(tableName).withScanFilter(scanFilter)
                    .withExclusiveStartKey(lastKeyEvaluated)
                    .withConsistentRead(true);
            scanResult = client.scan(scanRequest);
            if (scanResult.getCount() != 0) break;
            lastKeyEvaluated = scanResult.getLastEvaluatedKey();
        } while (lastKeyEvaluated != null);
        return scanResult.getItems();
    }
}

Tuesday, November 28, 2017

Accessing AWS cloud for test data


In most of the projects today, you see that code + data is stored in the cloud. Global players like Amazon, Microsoft and Google are offering their cloud services. As a test analyst, you might be required to write tests that access data from the cloud.

I faced a similar situation when the team I joined had their project in Amazon AWS. I was writing API tests using Rest Assured and I had to access the database for validation.

I am providing a sample, generic method here which people can use if faced with a similar situation.

First, from command line, you have to login to AWS, which saves the AWS token in your computer.
From your IDE ( I use IntelliJ Idea), go to terminal ( command line) and login to aws credentials.
Use your appropriate company name and user names.

$ -auth l
Username: l@.com
Password: ********


Select an account:
[0] -production-dev
[1] --preproduction-admin
[2] --production-poweruser
Selection: 0

Selected role: --production-dev
Saved AWS token ✔ (/Users//.aws/credentials [Profile=default]) 

I am using java and created the class called DatabaseAccess.java
I am using AP_SOUTHEAST_2, and you could use your own region where domain is hosted.
The table name is "SIT_Registrations" and you could use your own table here. The same with the registration Id. Remember we are using a scan here and we can get multiple values.

To further use it in your test, you would modify the method getRegistration from void to "ScanResult" and return the ScanResult to the calling method.

import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.model.*;
import java.util.HashMap;

public class DatabaseAccess {
    AmazonDynamoDB client;
    DynamoDB dynamoDB;

    public DatabaseAccess() {
        client = AmazonDynamoDBClientBuilder.standard().withRegion(Regions.AP_SOUTHEAST_2).build();
        dynamoDB = new DynamoDB(client);
    }

    public void getRegistration() {
        String tableName = "SIT_Registrations";
        HashMap scanFilter = new HashMap<>();
        String registrationId = "22343cc-bf10-4f7f-ab45-2e33dcasde7e";
        Condition condition = new Condition()
                .withComparisonOperator(ComparisonOperator.EQ.toString())
                .withAttributeValueList(new AttributeValue(registrationId));
        scanFilter.put("requestId", condition);
        ScanRequest scanRequest = new ScanRequest(tableName).withScanFilter(scanFilter);
        ScanResult scanResult = client.scan(scanRequest);
        System.out.println("Result: " + scanResult);
    }
}