Wikimedia Product/Analytics Infrastructure/Standard

The instrumentation platform is a set of interfaces that standardize the way we design and build software instrumentation. By using the instrumentation platform, software instruments can be controlled and coordinated, even across products. Through its guidelines and conventions, the instrumentation platform supports software re-use, and practices that make instrumentation more adaptable, rigorous, and safe.

It facilitates the creation of instrumentation events in response to situations of interest (such as an error or a button click) that are detected by software instruments. To assist in this process, an instrumentation platform library is provided for each product platform, with algorithms and an API regulated using a common standard. Developers have the ability to design their own instrumentation events using an event schema, collect events from one or more instruments together into different event streams, and manage those instruments remotely using an event stream configuration.

It is built using the event platform, which is more general system that also carries other kinds of event traffic, such as events used to drive application behavior.

Conformance
How conformance is determined. RFC 2199

Event
An event is a unit of structured data containing at least: The event will usually also contain additional data describing the event. Such data are specified by the type.
 * a time (at which the event occurred)
 * a type (specifying its intended data structure).

Instrumentation Event
An instrumentation event is an event carrying an observation about the software under instrumentation. They are strictly observational, and must follow a "prime directive": the software under instrumentation should behave identically whether or not the event is fired. This property allows instrumentation to be enabled or disabled at will, and ensures that regressions or interruptions in instrumentation do not degrade the actual software under instrumentation. This makes it safer for instrumentation to be managed independently, and allows systems which carry instrumentation events to be held to a lower service tier.

Instrument
An instrument is the unit of application code responsible for submitting the event data to the instrumentation platform library. The event data alone is not yet an event, as it does not have a time. The time, and other additional fields, will be added by the instrumentation platform library before the event is produced.

streams
.

Event
The event is formatted as a JSON string that can be validated against a corresponding JSONSchema event schema. The properties of an event must match a schema. The event will identify which schema it believes it matches by a schema URI given as the value of its  property. Instrumentation event schema descend from the instrumentation common schema. The fields of this common schema are filled out by the instrumentation platform library automatically, to make things more convenient for the engineer.

Properties can be added at any time before or after schema validation, but since schema validation is performed only once, properties added after validation may or may not conform to the schema. Some properties may be reserved, meaning that they are to be set exclusively by the software that controls a particular part of the event intake process. Tables of reserved properties are given below.

How event data changes during processing
The table below illustrates how a simple event that only carries a single string will look at each location in the process. A full description of the additional properties are given below.

Initialize
Binding of various things related to the integration layer, session change detection, etc.

Configure
Stream configuration loading.

Submit
Submit is the main algorithm of the library and is the only driver of runtime behavior post-initialization. Its steps are carried out in a particular order.

Three ways to submit
 * Submit buffered (input buffer)
 * Submit unbuffered (call submit directly)
 * Submit carbon copy (only done internally, always unbuffered)

Integration
The core algorithms make use of a number of platform-specific functions that are defined in the integration layer. These functions and their contracts are outlined below.