service-utils is a new NodeJS utilities library built to standardize NodeJS services at the Wikimedia Foundation, replacing service-runner, service-template-node, servicelib-node, and service-scaffold-node. It has a similar API to service-runner for easy migration.

It was built and currently maintained by User:TChin (WMF) under the Data Engineering Team to support the Event Platform, but is looking for shared ownership of the project.

Background/History

service-runner is a library built in 2015 to provide a common abstraction for all NodeJS services at the Wikimedia Foundation. At that time, all services were on bare-metal, and different teams used different libraries for logging and metrics. service-runner provided a way to build multiple services inside docker containers and spin them up in a worker cluster setup with restarts, rate limiting, etc. On top of that was service-template-node, which acted as a standardized framework for all NodeJS services to derive from, providing OpenAPI support, distributed tracing, standardized metrics, and some common helper functions.

As time went on, a few things happened.

1. The abilities of service-runner were superseded by Kubernetes. We do not need to roll our own cluster anymore, and all services that use service-runner now run with 0 workers.
2. service-runner has no maintainer, and uses libraries deprecated 7+ years ago.
3. service-runner supported statsd and Prometheus for metrics reporting. We standardized on Prometheus.
4. service-runner used bunyan and gelf-stream to export logs into logstash. We standardized on a common ECS logging format, which bunyan doesn't support, and logs into stdout in Kubernetes are automatically exported into logstash now.
5. service-template-node became a burden, as any update to the template required manual updates to every service that used the template. We don't have insight into how many services use the template, and the structure of services across teams very quickly fell out of lockstep.
   1. An attempt was made to fix in 2021 this via the project servicelib-node, which abstracted all the logic into different libraries that are then used in a lightweight scaffolding called service-scaffold-node. This would allow easy updating of the logic without touching the scaffold. However, team restructuring and deprioritization killed the initiative.

Because of all of these issues, when teams decide to build new NodeJS services, they had to decide between using the standardized but deprecated and ageing tooling, or put in the work to implement things themselves.

Concerns about the age of service-runner were raised every time services had to be upgraded to the latest NodeJS LTS version, and in 2024 to support the Event Platform's NodeJS services, service-utils was created.

Philosophy

The main goal of service-utils is to get services off of service-runner and by proxy service-template-node. It does not address the issue that service-template-node tried to solve of having a common, unified microservice structure. service-utils provides the tools, but leaves it up to the service implementor to use the tools correctly. For example, it provides helpers for distributed tracing, but does not automatically set it up for you. It supports ECS logging, but you have to make sure that your logs are formatted correctly.

In the future, a service-scaffold-node v2 could be built on top of it. The only thing service-utils doesn't (currently) do is provide an API wrapper for MediaWiki. However, since we don't create a lot of brand new NodeJS services anyways, it might be more effort than its worth.

Basic Setup

1. Install service-utils from Wikimedia's Data Engineering GitLab Package Registry:

   echo @wikimedia:registry=https://gitlab.wikimedia.org/api/v4/groups/189/-/packages/npm/ >> .npmrc
   npm i @wikimedia/service-utils
   

2. Create a config file service-utils.config.yaml. All keys are technically optional but at the bare minimum you should have a service_name:

   # service-utils.config.yaml
   service_name: foobar
   

3. Import service-utils in your codebase. service-utils exports a singleton through a getInstance() function. If you want to recreate the singleton (for example, with new config), you would need to call teardown() first.

   import { getInstance, teardown } from "@wikimedia/service-utils";
   const serviceUtils = await getInstance();
   
   // Somewhere else in the service
   serviceUtils.logger
     .log("info", "Test simple logger");
   
   // On service shutdown
   await teardown();
   

4. Run the NodeJS service normally. service-utils automatically looks for the config file in ${cwd}/service-utils.config

Configuration

service_name: default_service

logging:
  level: info
  format: ecs
  stacktrace: true
  transports:
    - transport: Console # Case matters

metrics: false

# service-utils.config.yaml
service_name: foobar
# Default config
logging:
  level: info
  format: ecs
  stacktrace: true
  transports:
    - transport: Console # Case matters

# Environment-specific variables, merged with defaults
# These can be arbitrarily named and applied on ServiceUtils instantiation with
# the `envName` option like: loadConfig({ envName: 'development' })
$development:
  service_name: foobar dev
  logging:
    level: verbose
    format: simple

$production:
  service_name: foobar prod
  logging:
    level: info
  metrics:
    port: 9001

# service-utils.config.yaml
service_name: some_service

foo: bar

import { getInstance } from "@wikimedia/service-utils";

interface CustomConfig {
  foo: string;
}

const serviceUtils = await getInstance<CustomConfig>();
// This will be typed
console.log(serviceUtils.config.foo);

Migrating from service-runner

service-utils is incrementally adoptable, and is easily migratable from service-runner. It is compatible with service-runner's config structure, and also maintains the -c cli args.

- bunyan.createLogger( { name: 'EventGate', src: true, level: 'info' } );
# Note that `name` != `service.name`, which is used in ECS format
+ winston.createLogger( { level: 'info', defaultMeta: { name: 'EventGate' } } );

- logger.trace({ event }, `Validating ${this.eventRepr(event)}...`);
+ logger.debug(`Validating ${this.eventRepr(event)}...`, { event });

function reqForLog(req, whitelistRE) {

-	const ret = {
-		url: req.originalUrl,
-		headers: {},
-		method: req.method,
-		params: req.params,
-		query: req.query,
-		body: req.body,
-		remoteAddress: req.connection.remoteAddress,
-		remotePort: req.connection.remotePort
-	};

+	const ret = {
+		source: {
+			ip: req.connection.remoteAddress,
+			port: req.connection.remotePort
+		},
+		url: {
+			full: req.originalUrl,
+			// service-template-node dynamically defines routes
+			// which are captured with params[n]
+			// https://expressjs.com/en/4x/api.html#req.params
+			path: req?.params?.[ 0 ] ?? req?.path ?? '',
+			query: req.query
+		},
+		http: {
+			request: {
+				method: req.method,
+				headers: {},
+				body: {
+					content: req.body
+				},
+				params: req.params
+			}
+		}
+	};

	if (req.headers && whitelistRE) {
		Object.keys(req.headers).forEach((hdr) => {
			if (whitelistRE.test(hdr)) {
-				ret.headers[hdr] = req.headers[hdr];
+				ret.http.request.headers[hdr] = req.headers[hdr];
			}
		});
	}

	return ret;

}

Links

• Service-runner
  • GitHub
• Service-template-node
  • GitHub
• Service-scaffold-node
• Service-utils GitLab
• Service-utils Phabricator Project