Using Java Protocol Buffers in JRuby

Written by Daniel Neighman and Matt Wilson.

At Square, we use Protocol Buffers for communication between services. Ruby has good native Protocol Buffer support, but by using the compiled Java classes, we can leverage all the benefits of any extensions and infrastructure relying on those extensions. We’ll demonstrate how we utilize Java-backed Protocol Buffers while still enjoying idiomatic Ruby for Protocol Buffer construction.

Protocol Buffers

We have added support to our JRuby-Java integration framework (Minecart) to seamlessly speak Square’s internal RPC protocol. If you are unfamiliar with the Protocol Buffer syntax, and you’d like to know more, please have a quick read. Protocol Buffers are a strongly typed message format which serialize to a binary representation. This results in an efficient wire transfer and documents the API. Furthermore, extensions can be built which add functionality such as idempotence, request chaining, and even behavioral requirements such as authentication for service endpoints.

To show off Protocol Buffers and how they work in Minecart, let’s implement part of a pizza ordering service. To start we’ll look at the service definition.

service OrderingService **{**
  rpc PlaceOrder **(** OrderRequest **)** returns **(** OrderResponse **)**
**}**

Great. This tells us that we have a service which can handle the action ‘PlaceOrder’. It requires an ‘OrderRequest’ message and returns an ‘OrderResponse’.

message OrderRequest **{**
  optional Person person **=** 1;
  optional Address address **=** 2;
  repeated LineItem line_items **=** 3;
**}**

message OrderResponse {
  optional string order_id = 1;
  optional bool success = 2;
  optional string error_message = 3;
}

Here we have the filled in OrderRequest and OrderResponse messages. These contain several nested messages which are defined similarly. You can find the filled out service definition here.

To get this service running in JRuby we’re going to install protoc, download the pizza service, download the Java dependencies, and then compile, install, and run.

brew install protobuf-c
mkdir protos
curl [https://gist.github.com/hassox/d4b3d65c9e00c116f208/raw](https://gist.github.com/hassox/d4b3d65c9e00c116f208/raw) > protos/pizza.proto
protoc --java_out**=**. --proto_path**=**protos/ protos/pizza.proto
curl -O [http://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar](http://repo1.maven.org/maven2/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar)
mkdir target
javac -classpath protobuf-java-2.4.1.jar com/pizza/Pizza.java -d target
*# For the future...*
curl [https://gist.github.com/hassox/e5f27688a783c1b43e7e/raw](https://gist.github.com/hassox/e5f27688a783c1b43e7e/raw) > hash_to_proto_converter.rb

BOOM! Now you we have access to the raw Java Protocol Buffers classes, give it a shot!

*#JRuby console*
require 'protobuf-java-2.4.1'
$CLASSPATH **<<** File**.**join(Dir**.**pwd, "target")

order_builder = com.pizza.Pizza::OrderRequest.newBuilder

person_builder = order_builder.getPersonBuilder
person_builder.setName   "Homer J. Simpson"
person_builder.setEmail  "[email protected]"
person_builder.setPhone  "555 1234"

address_builder = order_builder.getAddressBuilder
address_builder.setStreet1      "742 Evergreen Terrace"
address_builder.setCity         "Springfield"
address_builder.setState        "Unknown"
address_builder.setPostalCode   "12345"
address_builder.setCountry      "USA"

line_item_builder = com.pizza.Pizza::LineItem.newBuilder
[
  {name: "Peperroni Pizza", quantity: 1},
  {name: "Cheese Pizza", quantity: 8}
].each_with_index do |item, idx|
  line_item_builder.setItem  item[:name]
  line_item_builder.setQuantity  item[:quantity]
  order_builder.addLineItems(line_item_builder.build)
end

order = order_builder.build

You’ll notice that we have to set each field individually, and use a builder for nested types and enum values. Though we have gained access to any Protocol Buffer extensions, the cost is noticeable when compared to native Ruby Protocol Buffer support.

*# Alternative Ruby example using Ruby support for protos*
  Pizza**::**OrderRequest**.**new(
    person: {name: "Homer J. Simpson", email: "[email protected]"}
    **.**.**.**
  )

Enter the Java proto to Ruby Hash converter

To bring back delight and ease of use, we built a helper to interoperate Java Protocol Buffers and Ruby hashes. Note: we downloaded the converter previously during the setup so you should have access to it.

*# JRuby*
require 'protobuf-java-2.4.1'
$CLASSPATH **<<** File**.**join(Dir**.**pwd, "target")

require 'hash_to_proto_converter'

proto = Minecart::HashProtoBuilder.hash_to_proto(
    com.pizza.Pizza::Person,
    name: 'Fred',
    email: '[email protected]',
    phone: '555 1234'
  )

Minecart::HashProtoBuilder.hash_from_proto(proto)
# => { name: "Fred", email: "[email protected]", phone: "555 1234"}

As a developer you get to specify the contract, have a consistent view of the APIs and keep terse hash syntax.

A peek behind the curtain: this utility has allowed us to write Ruby handlers which tie seamlessly into our Java request lifecycle and infrastructure.

**class** **OrderingService** **<** Minecart**::**Service(com**.**pizza**.**Pizza**::**OrderingService)

  # Dispatch is mapped by snake cased name defined in the service proto
  def place_order
    params # the order Protocol Buffer Message, deserialized as a nested hash
    # snip… process the order
    { order_id: order.id, success: true } # return a hash that maps to the response message
  end
end

Beautiful!

This post is part of a series, which highlights discoveries and insights found while integrating Ruby with our robust Java stack. Daniel Neighman - Profile *Australia, for a long time now, has taken 'illegal immigrants' and sent them to 'offshore processing centers' (or…*medium.com Matt Wilson *Follow the latest activity of Matt Wilson on Medium to see their stories and recommends.*medium.com