Tutorial: Using Google RPC/ProtocolBuffers for Remote Services

Introduction

The ECF project provides an implementation of the OSGi R6 Remote Services and Remote Service Admin specifications. The RSA specification defines two major subsystems: discovery and distribution. Discovery concerns finding remote services exported by other processes on the network. The distribution subsystem is responsible for the actual communication of invoking a remote call: serializing remote method parameters, communicating with the remote service host via some network transport protocol, unmarshalling and invoking the service method with provided parameters, and returning a result to the consumer.

ECF's implementation of RSA defines an API to create new distribution providers. Recently, a distribution provider based upon Google RPC/Protocol Buffers (grcp) was created. This distribution provider allows any grcp service to be exported and/or imported using the OSGi RS/RSA standards.

This tutorial will present an example of using this provider.

Defining the service with grpc code generation

An important part of grpc's behavior is that it takes as input a protocol buffers service specification (.proto file), and generates code (in appropriate language) for the service implementation and the use of a stub by the remote service consumer. Here's an example (from grpc examples), of a helloworld proto file:

syntax = "proto3";

option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";

package helloworld;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}

This proto file is then run through the grpc/protocol buffers compiler to generate the code to support the serialization of HelloRequest and HelloReply (via Google's protocol buffers), and the code to support implementing the Greeter sayHello service implementation, and the GreeterService stub for use by the service consumer.

This example project contains the helloworld.proto file (in proto directory), and all of the grpc-generated source code.

Implementing the Greeter Service

Part of the function of the grpc code generation is to create an abstract superclass that can be used to create the service implementation. In the case of the Greeter service, this is the AbstractGreeter innerclass of the generated GreeterGrpc class. To implement the GreeterService, all that's necessary is to extend the AbstractGreeter class. For example:

package org.eclipse.ecf.examples.provider.grpc.helloworld.host;

import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;

import org.osgi.service.component.annotations.Component;

import io.grpc.examples.helloworld.GreeterGrpc.AbstractGreeter;
import io.grpc.examples.helloworld.GreeterService;
import io.grpc.stub.StreamObserver;

@Component(immediate = true, service = GreeterService.class, property = {
		"service.exported.interfaces=io.grpc.examples.helloworld.GreeterService",
		"service.exported.configs=ecf.grpc.server",
		"ecf.grcp.server.urlContext=http://localhost:50001" })
public class GreeterImpl extends AbstractGreeter implements GreeterService {

	@Override
	public void sayHello(HelloRequest req, StreamObserver<HelloReply> responseObserver) {
		// Call synchronous version of method to get reply
		HelloReply reply = sayHello(req);
		responseObserver.onNext(reply);
		responseObserver.onCompleted();
	}

	@Override
	public HelloReply sayHello(HelloRequest req) {
		// Respond with HelloReply with message "Hello there <name"
		return HelloReply.newBuilder().setMessage("Hello there " + req.getName()).build();
	}
}

The entire project containing this host implementation class is available here. The two versions of the sayHello method are implemented in terms of one another, and define the behavior of this remote service. In this case, the name provided in the HelloRequest is used to respond back with a HelloReply with a message of "Hello there <name>".

Exporting the GreeterService as an OSGi Remote Service

To export this grpc service as an OSGi Remote Service is this annotation in the class above

@Component(immediate = true, service = GreeterService.class, property = {
		"service.exported.interfaces=io.grpc.examples.helloworld.GreeterService",
		"service.exported.configs=ecf.grpc.server",
		"ecf.grcp.server.urlContext=http://localhost:50001" })
public class GreeterImpl extends AbstractGreeter implements GreeterService {

The @Component annotation is an OSGI-standard annotation for the Declarative Services part of the OSGi specification. The properties given are standard properties defined by the Remote Services part of the OSGi specification that tell the ECF RSA implementation to export this service via the ecf.grpc.server provider, and with hostname=localhost and port=50001. Everything about the grcp server setup/lifecycle, and the service export is taken care of by the grcp provider.

The GreeterImpl class is the only class needed to implement a fully functioning server and Greeter remote service. The complete project for the remote service host is available here.

Running the GreeterService Server

When the host is run here is the debug console output

osgi> 10:01:33.337;EXPORT_REGISTRATION;exportedSR={io.grpc.examples.helloworld.GreeterService}={component.name=org.eclipse.ecf.examples.provider.grpc.helloworld.host.GreeterImpl, ecf.grcp.server.urlContext=http://localhost:50001, component.id=1, service.exported.configs=ecf.grpc.server, service.exported.interfaces=io.grpc.examples.helloworld.GreeterService, service.id=54, service.bundleid=20, service.scope=bundle};cID=URIID [uri=http://localhost:50051];rsId=1
--Endpoint Description---
<endpoint-descriptions xmlns="http://www.osgi.org/xmlns/rsa/v1.0.0">
  <endpoint-description>
    <property name="component.id" value-type="Long" value="1"/>
    <property name="component.name" value-type="String" value="org.eclipse.ecf.examples.provider.grpc.helloworld.host.GreeterImpl"/>
    <property name="ecf.endpoint.id" value-type="String" value="http://localhost:50051"/>
    <property name="ecf.endpoint.id.ns" value-type="String" value="ecf.namespace.grpc"/>
    <property name="ecf.endpoint.ts" value-type="Long" value="1462986091915"/>
    <property name="ecf.grcp.classname" value-type="String" value="io.grpc.examples.helloworld.GreeterGrpc"/>
    <property name="ecf.grcp.server.urlContext" value-type="String" value="http://localhost:50001"/>
    <property name="ecf.rsvc.id" value-type="Long" value="1"/>
    <property name="endpoint.framework.uuid" value-type="String" value="708d9a00-9a17-0016-1988-c38915abd720"/>
    <property name="endpoint.id" value-type="String" value="726e58b3-a393-4e5f-a76c-b15a47e77001"/>
    <property name="endpoint.service.id" value-type="Long" value="54"/>
    <property name="objectClass" value-type="String">
      <array>
        <value>io.grpc.examples.helloworld.GreeterService</value>
      </array>
    </property>
    <property name="remote.configs.supported" value-type="String">
      <array>
        <value>ecf.grpc.server</value>
      </array>
    </property>
    <property name="remote.intents.supported" value-type="String">
      <array>
        <value>passByValue</value>
        <value>exactlyOnce</value>
        <value>ordered</value>
      </array>
    </property>
    <property name="service.imported" value-type="String" value="true"/>
    <property name="service.imported.configs" value-type="String">
      <array>
        <value>ecf.grpc.server</value>
      </array>
    </property>
  </endpoint-description>
</endpoint-descriptions>
---End Endpoint Description

This output shows that the GreeterService was successfully exported (EXPORT_REGISTRATION).

Consuming the Remote Service

On an OSGi consumer, using DS it's possible to have the GreeterService discovered and dynamically injected into application code. For example, here's an example helloworld consumer

package org.eclipse.ecf.examples.provider.grpc.helloworld.consumer;

import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;
import org.osgi.service.component.annotations.ReferencePolicy;

import io.grpc.examples.helloworld.GreeterService;
import io.grpc.examples.helloworld.HelloRequest;

@Component(immediate = true)
public class GreeterComponent {

	private GreeterService greeterService;

	@Reference(policy = ReferencePolicy.DYNAMIC)
	void bindGreeterService(GreeterService greeter) {
		this.greeterService = greeter;
	}

	void unbindGreeterService(GreeterService greeter) {
		this.greeterService = null;
	}

	@Activate
	void activate() throws Exception {
		System.out.println("Calling Greeter Service");
		System.out.println("Greeter service reply message="
				+ this.greeterService.sayHello(HelloRequest.newBuilder().setName("Scott").build()).getMessage());
	}
}

This GreeterComponent class is the only java code required. Using ECF RSA (including network discovery), the GreeterService will be discovered, a proxy created by the ecf.grpc.client provider, and then dynamically injected (via DS and bindGreeterService) into this application code. Then the greeter service is called to respond to the name "Scott". Note that there is no need to define a hostname and port for the client to connect to, as all of that information is present in the EndpointDescription, which can be dynamically discovered via the desired discovery protocol (etcd, zeroconf, slp, etc).

Running the Consumer

When run, the consumer prints out the following to the console

osgi> May 11, 2016 10:02:05 AM io.grpc.internal.ManagedChannelImpl <init>
INFO: [ManagedChannelImpl@596b2939] Created with target localhost:50001
Calling Greeter Service
Greeter service reply message=Hello there Scott

What happens is that the GreeterService is discovered via zeroconf and ECF's RSA impl, the ecf.rpc.client provider uses grpc to connect to the server, creates the GreeterService proxy, and this proxy is then injected into the consumer application code via Declarative Services. Once injected the activate method is executed, and this calls the sayHello method remotely (with serializing/deserializing HelloRequest/HelloReply via protocol buffers) and the HelloReply message is printed to the console.

The grcp provider bundle is available here and in the build directory is a pre-built version of this bundle.

The three example bundles described above (org.eclipse.ecf.provider.grpc.helloworld, org.eclipse.ecf.provider.grpc.helloworld.host, org.eclipse.ecf.provider.grpc.helloworld.consumer) are located here.

Note that using grpc and protocol buffers means that other languages/clients can consume this same remote service. All that's needed is that the service proto be compiled for other grpc-supported languages.