Event Platform Stream and Schema Index

This page provides a glimpse into Event Platform streams and schemas deployed in production. Source code is available at github.com/bearloga/wmf-mep-index

Repositories used (as submodules) and their hashes:

933cec2c2cfae7d76cffdab8c9b0ae2ffff9fcca mw-config (remotes/origin/HEAD) 56defa2c38c541180f3a5d2ddc83367ec45a4aeb repos/primary (remotes/origin/HEAD) 9ec0c485b2963dbcfe35632228fbe60b6de3764a repos/secondary (remotes/origin/HEAD)

Streams

Index of streams deployed in production (defined in $wgEventStreams in wmf-config/InitialiseSettings.php within mediawiki-config), via EventStreamConfig extension:

Stream Schema
eventlogging_SearchSatisfactionanalytics/legacy/searchsatisfaction
eventlogging_TemplateWizardanalytics/legacy/templatewizard
eventlogging_Testanalytics/legacy/test
test.instrumentationanalytics/test
test.instrumentation.sampledanalytics/test
mediawiki.client.errormediawiki/client/error
kaios_app.errormediawiki/client/error
api-gateway.requestapi-gateway/request
mediawiki.api-requestmediawiki/api/request
mediawiki.cirrussearch-requestmediawiki/cirrussearch/request
wdqs-internal.sparql-querysparql/query
wdqs-external.sparql-querysparql/query
cqs-external.sparql-querysparql/query
/^swift\.(.+\.)?upload-complete$/swift/upload/complete
/^mediawiki\.job\..+/mediawiki/job
mediawiki.centralnotice.campaign-changemediawiki/centralnotice/campaign/change
mediawiki.centralnotice.campaign-createmediawiki/centralnotice/campaign/create
mediawiki.centralnotice.campaign-deletemediawiki/centralnotice/campaign/delete
mediawiki.page-createmediawiki/revision/create
mediawiki.page-deletemediawiki/page/delete
mediawiki.page-links-changemediawiki/page/links-change
mediawiki.page-movemediawiki/page/move
mediawiki.page-properties-changemediawiki/page/properties-change
mediawiki.page-restrictions-changemediawiki/page/restrictions-change
mediawiki.page-suppressmediawiki/page/delete
mediawiki.page-undeletemediawiki/page/undelete
mediawiki.recentchangemediawiki/recentchange
mediawiki.revision-createmediawiki/revision/create
mediawiki.revision-scoremediawiki/revision/score
mediawiki.revision-tags-changemediawiki/revision/tags-change
mediawiki.revision-visibility-changemediawiki/revision/visibility-change
mediawiki.user-blocks-changemediawiki/user/blocks-change
resource_changeresource_change
resource-purgeresource_change
change-prop.transcludes.resource-changeresource_change
test.eventtest/event
eventgate-logging-external.test.eventtest/event
eventgate-analytics-external.test.eventtest/event
eventgate-analytics.test.eventtest/event
eventgate-main.test.eventtest/event
eventgate-logging-external.error.validationerror
eventgate-analytics-external.error.validationerror
eventgate-analytics.error.validationerror
eventgate-main.error.validationerror

Reminder: streams map to tables in the modern event platform.

Schemas

Flattened index of schemas from the primary and secondary schema repositories, as an alternative to schema.wikimedia.org:

Repository Schema $id (latest) Description
secondary/analytics/legacy/searchsatisfaction/1.1.0Tracks the dwell time and bounce rate of a user on pages linked from a search engine result page. Legacy metawiki schema: https://meta.wikimedia.org/wiki/Schema:SearchSatisfaction
secondary/analytics/legacy/templatewizard/1.0.0Tracking usage of the TemplateWizard extension.
secondary/analytics/legacy/test/1.1.0Test schema for checking that EventLogging works Legacy metawiki schema: https://meta.wikimedia.org/wiki/Schema:Test
secondary/analytics/session_tick/1.0.0Ticks sent at a pre-specified interval during an active client session. Used to compute a privacy-conscious session length metric. See: https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/SessionLength
secondary/analytics/test/1.0.0For simple tests of analytics instrumentation
secondary/fragment/analytics/app_identifiers/1.0.0Standard identifiers for Wikimedia product/feature analytics in mobile apps which use an implementation of the Event Platform Client (EPC) specification.
secondary/fragment/analytics/common/1.0.0Common fields for all WMF analytics and instrumentation events
secondary/fragment/analytics/legacy/eventcapsule/1.1.0A wrapper around (legacy) EventLogging event schemas that encodes generic metadata. Legacy metawiki schema: https://meta.wikimedia.org/wiki/Schema:EventCapsule
secondary/fragment/analytics/web_identifiers/1.0.0Standard identifiers for Wikimedia product/feature analytics on MediaWiki-powered mobile and desktop websites. NOTE: Unlike the fields in the related app_identifiers fragment -- which are filled in automatically by the Event Platform Client libraries for Android and iOS -- these fields are (currently) *not* automatically set by EventLogging's `mw.eventLog.submit()` and must be set by instrumentation code when it is putting together the event data.
secondary/sparql/query/1.0.0A sparql query received on the triplet store backends (e.g. query.wikidata.org)
secondary/swift/upload/complete/1.0.0Schema used for simple tests
primary/api-gateway/request/1.0.0A single request to the API Gateway
primary/change-prop/continue/1.0.0Represents a continuation request in batch processing
primary/change-prop/retry/1.0.0Represents a failure in event processing and a request to retry it
primary/error/1.0.0Represents an error event
primary/fragment/common/1.1.0Common schema fields for event schemas
primary/fragment/http/1.1.0Information about an HTTP request.
primary/fragment/mediawiki/common/1.0.0Common schema fields for all Mediawiki entities
primary/fragment/mediawiki/page/common/1.0.0Common schema fields for all Mediawiki page entities
primary/fragment/mediawiki/revision/common/1.0.0Common schema fields for all Mediawiki revision entities
primary/mediawiki/api/request/0.0.1A single request to the MediaWiki Action API (api.php)
primary/mediawiki/centralnotice/campaign/change/1.0.0Represents a change in the settings of a MW CentralNotice banner campaign
primary/mediawiki/centralnotice/campaign/create/1.0.0Represents the creation of a MW CentralNotice banner campaign
primary/mediawiki/centralnotice/campaign/delete/1.0.0Represents the deletion of a MW CentralNotice banner campaign
primary/mediawiki/cirrussearch/request/0.0.1A set of requests made by CirrusSearch to ElasticSearch for a single PHP execution context.
primary/mediawiki/client/error/1.0.0An error generated in a client runtime.
primary/mediawiki/job/1.0.0Represents a job in Mediawiki JobQueue.
primary/mediawiki/page/delete/1.0.0Represents a MW Page Delete event.
primary/mediawiki/page/move/1.0.0Represents a MW Page Move event.
primary/mediawiki/page/properties-change/1.0.0Represents a MW Page Properties Change event.
primary/mediawiki/page/restrictions-change/1.0.0Represents a MW Page Restrictions Change event.
primary/mediawiki/page/undelete/1.0.0Represents a MW Page Undelete (restore) event.
primary/mediawiki/recentchange/1.0.0Represents a MW RecentChange event. https://www.mediawiki.org/wiki/Manual:RCFeed
primary/mediawiki/revision/create/1.1.0Represents a MW Revision Create event.
primary/mediawiki/revision/score/2.0.0Represents a MW Revision Score event (from ORES).
primary/mediawiki/revision/tags-change/1.0.0Represents a MW revision tags change event
primary/mediawiki/revision/visibility-change/1.0.0Represents a MW Revision Visibiilty Change event.
primary/mediawiki/user/blocks-change/1.1.0Represents a MW User Blocks Change event
primary/resource_change/1.0.0Represents a change in a resource tied to the specified URI
primary/test/event/1.0.0Schema used for simple tests