JOSS Tutorial: using JOSS to access OpenStack Storage

In Accessing OpenStack Object Storage with Java through JOSS, Robert Bor introduced the JOSS library. JOSS allows developers to connect their Java applications to OpenStack Storage. This article demonstrates how.

Instant Developer Experience


Let’s start with a simple scenario: a Hello World style standalone application with an Instant Developer Experience (we’ve blogged before on this subject). During this blog post we’ll extend it to use JOSS. For a standalone application, I use the following POM:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>nl.tweeenveertig.openstack</groupId>
<artifactId>tutorial-joss-quickstart</artifactId>
<version>1.0</version>

<properties>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<dependencies>
<dependency>
<groupId>nl.tweeenveertig.openstack</groupId>
<artifactId>joss</artifactId>
<version>0.3.0</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<configuration>
<mainClass>nl.tweeenveertig.openstack.tutorial.MainClass</mainClass>
</configuration>
</plugin>
</plugins>
</build>
</project>

There’s not much to see here: just Java 6 with UTF-8 sources and a dependency on JOSS. Oh, and I’ve configured a plugin to run our application. This last bit is the only thing needed to create an instant developer experience here. As you can see, the assumption is that the entry point is a main class named nl.tweeenveertig.openstack.tutorial.MainClass. Let’s create it, and add this method:
public static void main(String… arguments) {
System.out.println("Hello World!");
}

We can verify it works by issuing the following command:
mvn compile exec:java

Logging in


Now we can add code to use JOSS. To start using JOSS we must login. We’ll add a file with our credentials, named src/main/resources/credentials.properties:
auth_url=https://os.cloudvps.com/v2.0/tokens
tenant=
username=
password=

The auth_url property is set for the CloudVPS OpenStack (their Keystone server), but you should provide your own. Also fill in the other properties using your account credentials with your OpenStack Storage provider.

The next step is to login. There’s one important bit between the initialization and informational fluff, as highlighted below:
ResourceBundle credentials = ResourceBundle.getBundle("credentials");
String tenant = credentials.getString("tenant");
String username = credentials.getString("username");
String password = credentials.getString("password");
String auth_url = credentials.getString("auth_url");
Account account = new ClientImpl().authenticate(tenant, username, password, auth_url);

System.out.printf("Account summary: %d containers containing %d objects with a total of %d bytes%n",
account.getContainerCount(), account.getObjectCount(), account.getBytesUsed());
printMetadata(false, account.getMetadata());

The printMetadata method you see is this:
private static void printMetadata(boolean useExtraIndent, Map metadata) {
String indentString = useExtraIndent ? " " : "";

System.out.print(indentString);
if (metadata.isEmpty()) {
System.out.println("(there is no metadata)");
} else {
System.out.println("Metadata:");
for (Map.Entry entry : metadata.entrySet()) {
System.out.print(indentString);
System.out.printf(" %s: %s%n", entry.getKey(), entry.getValue());
}
}
}

If you run the code so far, you’ll see the application login and display some basic account information.

Accessing the storage content


Logging in may be required, it’s not exciting. Let’s add a few loops to display our storage contents:
for (Container container : account.listContainers()) {
boolean isPublic = container.isPublic();
System.out.printf("%nContainer: %s (%s, %d objects using %d bytes)%n", container.getName(), isPublic ? "public" : "private",
container.getObjectCount(), container.getBytesUsed());
printMetadata(false, container.getMetadata());

if (container.getObjectCount() > 0) {
System.out.println("Contents:");
for (StoredObject object : container.listObjects()) {
System.out.printf(" %s%n", object.getName());
if (isPublic) {
System.out.printf(" Public URL: %s%n", object.getPublicURL());
}
System.out.printf(" Type: %s%n Size: %s%n Last modified: %s%n E-tag: %s%n", object.getContentType(), object.getContentLength(),
object.getLastModified(), object.getEtag());
printMetadata(true, object.getMetadata());
}
}
}

Although the code above does not demonstrate it, you can get a single container/object by name.
The class Account has a method getContainer(String name) to accomplish this for a container.
For objects, the class Container has the method getObject(String name).
Note though, that you’ll always get a container/object. It may not exist (yet).

You can use the methods Container.exists() and StoredObject.exists() to test whether it already exist. If necessary, you can then create it using Container.create() or StoredObject.uploadObject(…). Finally, you can use the delete() method to obvious effect. Note that a container must be empty to be deleted.

On a side note: the Account, Container and StoredObject all implement the Active Record design pattern. So apart from creating and deleting them, all updates are automatically persisted.

Storing content


The code so far shows a detailed content list of your storage cloud, so it displays exactly nothing. Let’s change that by adding some content.

To add content, we need a container. We don’t have one yet, so we’ll create one like this. Also, we’ll make it public so we can access documents with our browser. The code for this is:
Container myContainer = account.getContainer("MyContainer");
if (!myContainer.exists()) {
myContainer.create();
myContainer.makePublic();
}

The if statement is not really needed, but explains the difference between creating and retrieving a container to store something in. Note that the name of a container is unique per tenant. Also, the container name cannot be longer than 256 characters (URL encoded). So a name of Course Docs (encoded: Course%20Docs) uses 13 characters. The same applies to stored objects, but with a limit of 1024.

With a container ready, let’s add an object. I’ve chosen to upload a file, but you can also upload a byte[] or provide an InputStream. This means that you can upload generated/transformed content without first storing it in a file.
StoredObject someFile = myContainer.getObject("cloud-computing");
someFile.uploadObject(new File("Cloud-Computing.jpg"));
System.out.println(someFile.getPublicURL());

The next step is to verify it works, by navigating your browser to the URL the code prints to standard output.

What’s that? Metadata


Sometimes, you just need a bit of extra information on a container or object. For this end, all objects (the account, containers and objects) all have metadata. The metadata consists of key-value pairs, both of which are Strings. Due to how metadata is handled (in the HTTP headers of the OpenStack Storage REST interface), you should now have more than 90 key-value pairs for an item, and the total byte length should not exceed 4kb (4096 bytes).

Setting metadata is as easy as this:
Map<String, Object> metadata = new HashMap<String, Object>();
metadata.put("Information", "Almost, but not quite, entirely unlike tea.");
someFile.setMetadata(metadata);

Determining what (if any) metadata to use is not as trivial. The easiest catch-all is not to use any metadata, as he list of standard information is quite comprehensive. See the initial retrieval code for details. If you examine its output, you’ll notice the E-tag looks suspiciously like an MD5 hashcode — it is, and JOSS calculates and verifies it for you when you do not use an InputStream/OutputStream to upload/download the stored objects.

Conclusion


That’s it! All you need to use JOSS.

For those who feel lazy, all code discussed here is available on GitHub as javaswift / tutorial-joss-quickstart.