AWS re:Invent Notes (1 of 3): SDK for Java

The following are my (only minimally edited) notes from the AWS re:Invent session (I highly recommend watching the recordings of the keynote sessions)

There are two usage levels in the SDK:

Low level access to AWS system APIs; and
Higher level facilities, that make it easier to access the underlying services.

Most notably, the SDK provides support for S3, DynamoDB, Policy API, Flow (SWF).

—
S3 TransferManager
Support async transfers
It is also a good example of hiding complexity from the developer, providing a clean interface.

Use ClasspathPropertiesFileCredentialsProvider#getCredentials to manage credentials (AWSCredentialsProvider interface)
One can also load credentials from the EC2 instance that is running the web app
There are a number of “providers” that cover most use cases, but if you don’t like any of those, you can always implement your own.

TransferManager manages the pool of connections and is thread-save, so it’s best used as a Singleton; the main method to use is upload( ) with a PutObjectRequest( )object, created on the fly.

The main stuff is in the Transfer i/f: very useful is the progress listener, which has one callback progressChanged(ProgressEvent ev)
ProgressEvent will give you info about bytesTransferred (and a bunch of other info)

You can hook the listener into the Request, get the progress from the Upload object and then display progress (eg in a progress bar).
—
S3 Encryption Client

Client-side, industrial-strength encryption facility, it is the one used by AWS internally.

AmazonS3EncryptionClient
Encrypts data on the fly, you can then use the standard S3 API, no other changes needed

The key management is a bit more complex than just straight encryption using a symmetric shared key, in order to make the whole process much more efficient and provide an easy way to update private keys should any one get compromised, without having to re-encrypt all the data.

Application holds the Data to encrypt and the MasterKey; the SDK generates a a one-time symmetric envelope key (see picture below): the Envelope Key is encrypted with the Master Key, while the data is encrypted with the Envelope Key.

Not only symmetric encryption with the Envelope Key is faster, but, should the Master Key be compromised, a new key can be generated, all the envelope keys decrypted with the old (now compromised) one and re-encrypted with the new (secure) one.

Clearly this process does not require any change to the encrypted data (whose size is likely to be much bigger than the envelope key) and thus much more efficient.

For more details see the article here.

—

DynamoDB

(Note this is very similar to Android SDK, see further below)

AmazonDynamoDBClient(credentialsProvider)

It is sometimes useful to versionize the entities saved on DynamoDB so as to avoid concurrent changes:

@DynamoDBVersionAttribute
public Long getVersion( ) {
return version;
}

Will throw when there are concurrent changes on the same object done by some other thread
The SDK is compatible with Android SDK: so the same code, with the same annotations will “just work” when used in an Android client.

It also supports batch updates

—
The SDK uses Apache Commons logging, so it’s really easy to view client logs Eclipse’s console by turning on ‘wire logging’ in log4j.properties:

log4j.logger.httpclient.wire=DEBUG

For (much) more detail, see the article here.

AWS SDK for Android

In addition to the AWS Android SDK, to enable support for dynamoDb, just add the relevant jar (installing the SKD via Eclipse will do it for you).

AmazonDynamoDBClient is the main class, along with the AttributeValue one

Use the DyanmoDBMapper to support simple CRUD operations, as it provides both query/scan capabilities to find items and batch operations (but those are not transactional, so can have ‘partial success’)

DynamoDB support in Java is modeled after JPA, in that it uses annotations (but it does not conform to JPA annotations):

@DynamoDBTable
public class UserPreferences {
….
}

// to define the hash key for the table

@DynamoDBHashKey
public int getUserId() {
return id;
}

@DynamoDBAttribute
// not strictly necessary for ‘standard’ getter/setters

// as they will be automatically marked for inclusion
public String getLastName() {
return lastName;
}

@DynamoDBIgnore
// for fields we don’t need to persist (or are computed)

@DynamoDBVersionAttribute
// flag one attribute as a version number
To track if data was changed while we were trying to save it, and an Exception will be thrown is another client has updated the row since we last read it, and when we attempt to push an update.

For fields that cannot be trivially serialized or need custom treatment before being written to the db, use `marshalling`:

@DynamoDBMarshalling(marshallerClass = Conv.class)
public ComplexType getComplexValue() { return myComplexValue; }

Then define a class that implements the DynamoDbMarshaller interface:

public class Conv implements DynamoDbMarshaller {
public String marshall(ComplexType val) {

// do something with val and convert to string

return valAsString;

}
public ComplexType unmarshall(Class clazz, String complexAsString) {

// do something like:

ComplexType res = ComplexType.parse(complexAsString);

return res;

}

Read/Write behavior — supports eventual consistency
Table Override — supports dev/prod environments

Code Trips & Tips

Adventures in developing distributed systems in Java and Python