The Binder Shell is a POSIX-compatible shell (though there are some missing features, such as pipes), which includes special extensions for interacting with the Binder. The shell serves as an example of another language that can work with the Binder runtime, and you can even write Binder components with the shell.
Shell commands themselves are Binder components. That is, when executing a command, the shell finds and instantiates a Binder component that implements the specified command. This allows the shell and the commands to negotiate extensively about what process the command will run in for example, a command can request that it run in a specific process next to an associated server, or the shell can allow some or all commands to run in its own process to improve performance.
The vast majority of the Binder system is accessible through the shell, due to the underlying scripting protocol supported by the Binder. For example, you can access the persistent data store, running services, and Binder components. Because of this, the shell provides a useful way to introduce many of the Binder concepts. All of the operations described in this tutorial map directly to corresponding calls in the underlying C++ APIs.
Note that the Binder Shell is not currently a replacement for a Linux command shell. In particular, it currently is not able to execute traditional Linux commands (via fork() and exec()), nor does it provide access to the underlying Linux filesystems. These are not design limitations of the shell, and we would like to add these facilities in the future.
From within the Binder system, you have a number of programmatic options:
All of the examples here will assume you have gotten into the shell through the build system's "make runshell" command.
Once you have the shell running, the first command you will want to know about is "help". This command can can be used in four different ways:
The "CD" portion is the current directory of the shell, which usually starts out at the root of the namespace. As such, these are the exact shell prompts you will usually see.
/#
/$
>
/# ls contexts/ packages/ processes/ services/ settings/ who_am_i
What you are seeing here is the root of the Binder Namespace, called a "Binder Context". This is a global namespace of data and services, and all of the normal shell filesystem commands work on this namespace. The Binder Context is your connection to the rest of the Binder system it gives access to the information needed to instantiate components, find running services, etc.
/settings
, the "settings catalog". This is a repository of persistent data anything you place here will be put into persistent storage (currently as an XML file stored on the Linux filesystem), which is read back the next time smooved starts.
For example, we can use the publish
command to place a piece of data into the settings catalog:
/# publish /settings/vendor/foo bar
And then later retrieve that data:
/# lookup /settings/vendor/foo Result: "bar"
If we were to now kill smooved and restart it, we would see that our data was still there:
/# ls -l /settings/vendor foo -> "bar"
(Note that ls -l
in the Binder Shell displays the data that is stored at each entry listed.)
All data in the Binder must be language independent that is, it must carry type as well as value information, have a consistent representation, and be able to express complex types. This facility is provided by a typed data API called SValue.
An SValue is similar to a COM variant (or GLib GValue), in that it holds a typed piece of data, such as an integer, string, etc. In addition, SValue can also contain complex sets of mappings of any such data value (including other mappings). SValue also does not impose a fixed set of types, instead defining a generic representation of a type constant, length, and blob of data.
Unlike a standard POSIX shell, all data that we work with in the Binder Shell is an SValue. This includes environment variables, command arguments, and command results. Thus when we executed this command:
/# lookup /settings/vendor/foo Result: "bar"
What we were seeing is that the lookup command returned the data it found under /settings/vendor/foo
, which is an SValue containing the string "bar".
The default data type that you create in the shell is a string, to match the standard POSIX semantics. To create types of other values, you can use a new @{ }
shell syntax:
@{ 10 }
is an integer.@{ foo, bar }
is a set (here of strings).@{ foo->bar }
is a mapping (here of strings).@{ foo->10, 2->bar }
is a set of mappings (here of strings and integers).
Given that, we can use this with the publish
command to place more complex data into the settings catalog:
/# publish /settings/map @{ 10 -> "99" } /# lookup /settings/map Result: int32_t(10 or 0xa) -> "99"
@{ 10 -> "99" }
@{ }
syntax is extremely rich. Here is a brief summary of common things you can do with it:
/# F=@{ 1.0 } # Make a floating point number /# I=@{ [abcd] } # Make a 32 bit integer of ASCII chars /# B=@{ true } # Make a boolean /# S=@{ "1" } # Make a string /# I64=@{ (int64_t)10 } # Make a 64 bit integer /# T=@{ (nsecs_t)123456 } # Make a time /# B=@{ (bool)$I64 } # Convert to a boolean /# I=@{ (int32_t)$S } # Convert to an integer /# S=@{ (string)$B } # Convert to a string /# M1=@{ a->$S } # Make a mapping /# M2=@{ a->$B, b->"foo" } # Make set of mappings /# M=@{ $M1 + $M2 } # Combine the mappings /# V=@{ $M[a] } # Look up a value in a mapping
See Binder Shell Data for more details on the @{ }
syntax and SValue data types.
/# VAR=something Result: "something" /# VAR=@{ 10 -> "99" } Result: int32_t(10 or 0xa) -> "99"
In addition, command results are typed and the Binder Shell introduces a new $[ ]
syntax for easily retrieving the result of a command, much like the standard $( )
syntax for retrieving a command's output:
/# VAR=$[lookup /settings/vendor/foo] Result: "bar" /# echo $VAR bar
This facility will be used extensively later as we look at new commands that generate complex data results.
/services
, which is a standard catalog holding various active services:
/# ls /services informant memory_dealer tokens/
The data here is our new type of SValue, an object. For example, if we use the lookup
command to retrieve an entry in this catalog, this is what we will see (when called from the same process as the object):
/# lookup /services/informant Result: sptr<IBinder>(0x80966bc 9Informant)
Such an object is an active entity, holding both state and the implementation for manipulating that state.
Binder objects are reference counted the object will be destroyed for you when all references on it go away. Unlike the data we saw before, objects are passed by reference, even when going across processes. For example, if you call a shell command with an object as an argument, that command will be operating on the same object as the one you gave it. Normal data, in contrast, is copied, so the command could not modify the data being held by the caller.
Every Binder object provides a set of properties and methods that others can use. The Binder Shell commands get
and put
are used to retrieve and modify the properties on an object, respectively. The command invoke
is used to call a method on an object. For all of these commands, the first argument is the object to operate on either an actual object value, or a path in the namespace to an object.
/# invoke /services/informant Inform myMessage "This is some data"
Well, okay, that wasn't very thrilling when the system first starts up, nobody has registered to receive a notification, so there is not much we can do.
Recall, however, that we mentioned earlier that Binder Shell commands are Binder components... and that includes the Binder Shell itself. In fact, there is a special shell variable called $SELF
that provides the IBinder object for the current shell. This gives us a more interesting opportunity.
First, let's define a function in our current shell session:
/# function Receiver() { echo "Receiver: " $1 "," $2 "," $3; }
We can now use the $SELF
variable to register our shell with the informant, giving the name of our function as the method it should call:
/# invoke /services/informant RegisterForCallback "myMessage" $SELF "Receiver"
And now let's try again the first method invocation:
/# invoke /services/informant Inform myMessage "This is some data" /# Receiver: This is some data , myMessage ,
What we see here is that after calling Inform on the informant and returning, the Informant has then called back on our shell to tell it the message that was broadcast.
A look at this same kind of operation through the C++ APIs can be found in Binder Recipes.
interfaces/services/IInformant.idl
.If you are already familiar with COM or CORBA, one thing that is worth pointing out is that in the Binder these interfaces are not the core communication protocol. Instead, the standard Binder language (as defined by IBinder) is a dynamically typed scripting protocol, which the Binder Shell sits directly on top of. Interfaces, then, are just formalized specifications of a scripting protocol that an IBinder provides, but you do not need to know about a specific interface to operate on a Binder object.
We can now look at the informant IDL file to see just what we were doing to that service:
namespace palmos { namespace services { interface IInformant { methods: // flags are the IBinder link flags status_t RegisterForCallback(SValue key, IBinder target, SValue method, [optional]uint32_t flags, [optional]SValue cookie); ... status_t Inform(SValue key, SValue information); } } } // namespace palmos::services
You can use the inspect
command in the shell to find out about the interfaces that an object implements. For example, we can look at our informant service:
/# inspect /services/informant Result: "org.openbinder.services.IInformant" -> sptr<IBinder>(0x8096794 9Informant)
An interface can also implement multiple interfaces, in which case inspect
will return all of them:
/# inspect /processes Result: { "org.openbinder.support.INode" -> sptr<IBinder>(0x808b5a4 14ProcessManager), "org.openbinder.support.ICatalog" -> sptr<IBinder>(0x808b594 14ProcessManager), "org.openbinder.support.IIterable" -> sptr<IBinder>(0x808b5b4 14ProcessManager), "org.openbinder.support.IProcessManager" -> sptr<IBinder>(0x808b6b8 14ProcessManager) }
This brings us to another important use of inspect
, interface selection. You can supply a second parameter to inspect
that specifies an interface you would like; in this case inspect will return either the Binder object of that interface or undefined
if the interface is not implemented.
/# inspect /processes org.openbinder.support.IProcessManager Result: sptr<IBinder>(0x808b6b8 14ProcessManager)
Note that like the lookup
command, inspect
returns its result as a typed value. You can use the $[ ]
syntax we saw previously to retrieve that value.
ls
, publish
, etc) are simply making calls on these standard interfaces.
In other words, the entire Binder namespace is simply a hierarchy of active objects. Some of these are generic directories that we can place anything inside of (such as the /services
directory), but often these are special implementations of the namespace interfaces. For example, the Settings Catalog that we looked at previously provides its own implementation that keeps track of all the data placed into it and saves that data out to an XML file.
As an example, we can retrieve the INode interface for the /services
directory:
/# n=$[inspect /services org.openbinder.support.INode] Result: sptr<IBinder>(0x8091d74 8BCatalog)
One of the things that INode does is provide access to meta-data about the namespace entries, which we can now retrieve through direct calls on the interface:
/# get $n mimeType Result: "application/vnd.palm.catalog" /# get $n modifiedDate Result: nsecs_t(13107d 10h 59m 57s 746ms 551us or 0xfb7648f415af8d8)
It is not uncommon for a service to implement both an API specific to that service, as well as the generic data model interfaces. This allows it to publish parts of itself in a standard way that is easily accessible through the shell and everything else that operates on the namespace, while at the same time providing a more specialized interface for more complicated interactions.
org.openbinder
. In addition, for historical reasons components that are a part of the Binder runtime itself use the prefix palmos
.
A new component instance is created with the new
command, which takes the name of the component to instantiate and returns an IBinder object of the new instance.
/# s=$[new org.openbinder.samples.Service] Result: sptr<IBinder>(0x80a0194 13SampleService) /# invoke $s Test Result: int32_t(12 or 0xc)
These shell commands have instantiated a sample service component that is included with OpenBinder and called a method on it.
You can optionally supply a second argument to new
, which is a set of mappings providing arguments to the component constructor.
/# s=$[new org.openbinder.samples.Service @{start->500}] Result: sptr<IBinder>(0x80a19a4 13SampleService) /# invoke $s Test Result: int32_t(501 or 0x1f5)
An alternative syntax for this allows you to bundle the component name with its arguments as a single complex data mapping.
/# s=$[new @{ org.openbinder.samples.Service->@{start->500} }] Result: sptr<IBinder>(0x80a19a4 13SampleService) /# invoke $s Test Result: int32_t(501 or 0x1f5)
This last form can be very useful when providing a component name to another entity that will instantiate it it allows you to bundle any other additional data along with the component.
new
, you use the special command new_process
to create a new process object.
/# p=$[new_process my_process] LOCK DEBUGGING ENABLED! LOCK_DEBUG=15 LOCK_DEBUG_STACK_CRAWLS=1 START: binderproc #29572 my_process Result: IBinder::shnd(0x2)
The result we see here is a new kind of Binder object, a handle. Instead of being a pointer to an object in the local process, it is a descriptor to an object that lives in another process. Besides how it looks when printed, however, it operates and behaves just like the objects we have seen so far.
In fact, we can use the inspect
command to find out a little more about this process object we have:
/# inspect $p Result: "org.openbinder.support.IProcess" -> IBinder::shnd(0x2)
If you want, you can go and look at the IProcess IDL file. However, you generally won't use that interface directly, but rather use the process object with other APIs.
The /processes
catalog is used as a place where you can store references to processes that will be shared between components. The normal boot script included with OpenBinder uses this to publish a process called "background" that can be used for components that don't need a process of their own.
new_process
to create a new empty process (a sandbox) and then create components inside of it. You do this with the -r
option on new
, which tells new
which process you want the object created in. Let's go back to our original example of creating a component and now create it in our process.
/# s=$[new -r $p org.openbinder.samples.Service] Result: IBinder::shnd(0x3) /# invoke $s Test Result: int32_t(502 or 0x1f6)
Notice that besides the kind of object returned, our service works just like it did when we had it running in the local process. This is a very important characteristic of the Binder it completely hides the location of objects, making remote objects look and behave just like local objects. This gives us a lot of flexibility in deciding, even at run time, how objects will be distributed across processes.
When looking at the overall system, you can see where IPC will happen simply by looking at which process objects are located in, and which objects call on to each other. Communication between objects is always 1:1 once you transfer an object reference from one process to another, that will set up a direct communication channel between them and your own process will no longer be involved. Even if your own process completely goes away, the connection you set up between the other two processes will remain undisturbed.
/# p= Result: "" /# s= Result: "" /# EXIT: binderproc #29572 my_process [SIGCHLD handler] child process 29572 exited normally with exit value 0
The stop_process
command gives you more control over this behavior. It takes a process object as an argument, and will have the process go away as soon as all remote reference on just that process object have been removed.
/# p=$[new_process my_process] LOCK DEBUGGING ENABLED! LOCK_DEBUG=15 LOCK_DEBUG_STACK_CRAWLS=1 START: binderproc #29607 my_process Result: IBinder::shnd(0x2) /# s=$[new -r $p org.openbinder.samples.Service] Result: IBinder::shnd(0x3) /# stop_process $p /# p= Result: "" /# EXIT: binderproc #29607 my_process [SIGCHLD handler] child process 29607 exited normally with exit value 0 /# invoke $s Test Result: Unknown error (0x80004a03)
In addition, if you use the -f
flag with stop_process
then the process will go away immediately, no matter what references there are on it.
/# stop_process -f $PROCESS /# EXIT: binderproc #29568 background hackbod@hackbod:~/lsrc/open-source/openbinder/main/dist$
Here we used stop_process
with our own process to make it exit. The line about the "background" process exiting is because this also causes us to drop our reference on the background process that the boot script had created.
/# s=$[new org.openbinder.samples.ServiceProcess] LOCK DEBUGGING ENABLED! LOCK_DEBUG=15 LOCK_DEBUG_STACK_CRAWLS=1 START: binderproc #29634 /home/hackbod/lsrc/open-source/openbinder/ main/dist/build/packages/org.openbinder.samples.ServiceProcess Result: IBinder::shnd(0x3)
Here the component has asked that it be instantiated in a process that is dedicated to components in its package. When the first instance of the component (or other such components in this package) is created, that process is started and the component created inside of it. Additional instances of the component will be placed into the same process.
We can now see this process in the /process
catalog:
/# ls /processes /home/hackbod/lsrc/open-source/openbinder/main/dist/build/ packages/org.openbinder.samples.ServiceProcess background system
Note that the current process manager uses the path to the component implementation as a unique name for the process, however the process manager itself is just a component that can be customized to provide whatever behavior you desire.
If we make another instance of the service component, it will be created in the same process as the first one:
/# s2=$[new org.openbinder.samples.ServiceProcess] Result: IBinder::shnd(0x4)
Likewise other components in the package can also specify that they would like to run in the package's process:
/# s3=$[new org.openbinder.samples.ServiceProcess.Command] Result: IBinder::shnd(0x5)
Of course they don't need to do so, in which case they will be instantiated as normal, usually in the process of the caller.
As we saw before in Process Lifetime, this process will continue to remain around only for as long as it is actually needed:
/# s= Result: "" /# s2= Result: "" /# s3= Result: "" /# EXIT: binderproc #21330 /home/hackbod/lsrc/open-source/openbinder/ main/dist/build/packages/org.openbinder.samples.ServiceProcess [SIGCHLD handler] child process 21330 exited normally with exit value 0
For more on processes, see the Binder Process Model.
/# p=$[new_process secondary] LOCK DEBUGGING ENABLED! LOCK_DEBUG=15 LOCK_DEBUG_STACK_CRAWLS=1 START: binderproc #29658 secondary Result: IBinder::shnd(0x4) /# su -r $p Starting sub-shell in system context... /# v1=$[lookup /services/informant] Result: IBinder::shnd(0x7) /# v2=$[lookup /services/informant] Result: IBinder::shnd(0x7)
What we are seeing here is that when an object reference is transfered between processes (here from the smooved to the "secondary" process that we created), as long as the receiving process already knows about that object it will always get the same identity.
However, that object will have different identities in different processes:
/# exit /# v3=$[lookup /services/informant] Result: sptr<IBinder>(0x8096744 9Informant)
The Binder IPC Mechanism is responsible for correctly mapping between these identities as objects move between processes. In essence, this provides a capability-style security model for Binder objects you can only perform operations for which you have been granted explicit access through a Binder object, and you can always validate the identity of an object (against an existing reference to it) when you receive it.
ls
, lookup
, and other commands. A component is "instantiated inside" of a context that is, you use a context to instantiate a component, and that new component instance also uses your context.
There can be more than one context. The default boot script that we have been using with smooved creates two contexts, which you can see by looking at the /contexts
directory:
/# ls /contexts system/ user/
The system context is the one we have been in, and has complete access to all system resources. The user context is a more restricted environment, which is not able to do things like modify the contents of the /services
directory.
You can use the su
command to switch between contexts (and processes, as we saw earlier):
/# lookup who_am_i Result: "System Context" /# su user Starting sub-shell in new context... /$ lookup who_am_i Result: "User Context" /$ exit /# lookup who_am_i Result: "System Context"
The "who_am_i" entry is created by the boot script in each context, as a debugging aid.
new
command, providing the information about the component needed to find and instantiate it./packages
, and under that directory provides a hierarchy of information associated with packages. One of the most important things here is the components
sub-directory, which contains information about every component in the system:
/# ls /packages/components org.openbinder.samples.Component org.openbinder.samples.Service org.openbinder.samples.ServiceProcess org.openbinder.samples.ShellService org.openbinder.kits.support.CatalogMirror org.openbinder.services.MemoryDealer org.openbinder.services.Settings org.openbinder.services.TokenSource org.openbinder.services.base.Informant org.openbinder.tools.BinderShell org.openbinder.tools.BinderShell.Cat org.openbinder.tools.BinderShell.Clear ...
Under each of these entries is a block of data describing everything about the component.
/# lookup /packages/components/org.openbinder.tools.BinderShell Result: { "file" -> "/home/hackbod/lsrc/open-source/openbinder/main/dist/build/packages/org.openbinder.tools.BinderShell/BinderShell.so", "package" -> "org.openbinder.tools.BinderShell", "modified" -> int32_t(1132535954 or 0x43812092), "interface" -> "org.openbinder.tools.ICommand", "packagedir" -> "/home/hackbod/lsrc/open-source/openbinder/main/dist/build/packages/org.openbinder.tools.BinderShell" }
When the Package Manager is started, a set of queries on the package data is also created and published in its directory. These allow you to find components for various purposes. For example, the Binder Shell itself uses the /packages/bin
query to find the component that implements a particular shell command.
/# ls /packages/bin [ atom bperf cat clear components
/# lookup /packages/bin/cat Result: "org.openbinder.tools.BinderShell.Cat"
You can also do queries for components implementing specific interfaces and other data in the manifest.
The manifest is an XML file that provides all information to the package manager about the components in the package. Note that unlike COM, component information comes only from the static manifest file there is no need to register a component with the package manager, and the package manager is free to reconstruct its component information from scratch at any time from the package manifests.
A typical manifest file can be seen in the SampleComponent sample, which implements a Binder Shell command.
<manifest> <component> <interface name="org.openbinder.app.ICommand" /> <property id="bin" type="string">sample_component</property> </component> </manifest>
Here we can see that the package contains a single component, whose name is the same as the package name. That component implements the ICommand interface. The <property>
tag here adds an additional property, called "bin". This is the property the Package Manager looks for to construct the /packages/bin
query, and thus how you make the component visible to the Binder Shell.
Here is a more complicated manifest, from the ServiceProcess sample. This contains two components, both of which would like to run in their own process for the entire package. The second component is a shell command.
<manifest> <component> <process prefer="package" /> <interface name="org.openbinder.support.INode" /> <interface name="org.openbinder.support.IIterable" /> <interface name="org.openbinder.support.ICatalog" /> </component> <component local="Command"> <process prefer="package" /> <interface name="org.openbinder.app.ICommand" /> <property id="bin" type="string">service_command</property> </component> </manifest>
/# s=$[new org.openbinder.samples.Service] Result: sptr<IBinder>(0x809f86c 13SampleService)
There are actually two kinds of object pointers. The one we have seen so far is a "strong pointer" or "sptr" because it ensures that the object will remain valid.
The other type of reference is called a "weak pointer" or "wptr". You won't normally see these in the shell, however, we can use a cast to explicitly create a weak pointer:
/# w=@{(wptr)$s} Result: wptr<IBinder>(0x809f86c 13SampleService)
Unlike a strong pointer, an object is allowed to go away while others hold weak pointers to it. We can see this in action if we now clear the strong pointer:
/# s= Result: ""
What happens to our weak pointer in this case is a little subtle. If we try to invoke a method on that, it will fail, because the object is no longer valid:
/# invoke $w Test invoke: invalid target object ''. Not going to do an invoke. Result: Unknown error (0x80000502)
However, the weak pointer itself still needs to hold something, so if we print it we will see that it still has a valid pointer, even though the object is no longer usable:
/# echo $w SValue(wptr<IBinder>(0x809f86c))
Notice the subtle point that though we are still seeing a valid address printed, we no longer see the actual class name like we did before.
If we now try to cast that weak pointer, we will see that this fails:
/# s=@{(sptr)$w} Result: sptr<IBinder>((nil))
This is, in fact, what happened when we tried to use invoke
it implicitly tries to convert its input to a strong pointer so it can call it, which fails. In the C++ APIs, this conversion is done explicitly through the wptr<>::promote() method.
Note that all of these rules for weak pointers hold for processes as well as objects. That is, a process's lifetime is determined by strong pointers on its objects holding a weak pointer on an object, including the IProcess object itself, will not prevent the process from going away.
There are two kinds of links an object can push: properties and methods. Properties can only be linked to other properties, and events can only be linked to methods. This restriction is because the scripting protocol for properties and methods is different.
You will generally find out about what you can link to by looking at an interface's IDL file. As an example, let's look at the interface for INode, one of the data model interfaces. It can be found in interfaces/support/INode.idl. The thing we are interested in is this event declared towards the end:
namespace palmos { namespace support { interface INode { ... events: ... // This event is sent when a new entry appears in the catalog. "who" is the parent node in which this change occured. "name" is the name of the entry that changed. "entry" is the entry itself, usually either an INode or IDatum. void EntryCreated(INode who, SString name, IBinder entry); ... }
Given that, let's write a shell function that handles the same method signature:
/# function handleEntryCreated() { echo "Created: parent=" $1 echo "Created: name=" $2 echo "Created: entry=" $3 }
We can now use the link
shell command to set up a link from the /services
directory to our new shell method.
/# n=$[inspect /services org.openbinder.support.INode] Result: sptr<IBinder>(0x8091dc4 8BCatalog) /# link $n $SELF @{EntryCreated->handleEntryCreated}
This says "make a link from the object $n
to the object $SELF
, such that when EntryCreated
is pushed from $n
we will have the handleEntryCreated
method called on $SELF"
. And thus, upon adding a new entry to /services
, we will see this:
/# publish /services/test linktest Publishing: /services/test /# Created: parent= SValue(sptr<IBinder>(0xb69063f4 8BCatalog)) Created: name= test Created: entry= SValue(sptr<IBinder>(0x8081254 N18SDatumGeneratorInt12IndexedDatumE))
Links are a very powerful mechanism, though not the only way to achieve the same result. Depending on your needs, you can just as well write your own notification mechanism for a specialized purpose, or use IInformant for a more generalized implementation of broadcasting without using links. A plain link, however, has significant advantages in being clearly documented in IDL and a standard mechanism that many other things will be able to use without being written specifically to receive your event.