Wikimore

Lua

Code Discussion Edit History Links Link count Subpages:Documentation Tests Results Sandbox Live code All modules

UnitTests provides a unit test facility that can be used by other scripts using require. See Wikipedia:Lua#Unit_testing for details. Following is a sample from Module:Bananas/testcases:

-- Unit tests for [[Module:Bananas]]. Click talk page to run tests.
local p = require('Module:UnitTests')

function p:test_hello()
    self:preprocess_equals('Hello, world!', 'Hello, world!')
end

return p

The talk page Module talk:Bananas/testcases executes it with {{#invoke: Bananas/testcases | run_tests}}. Test methods like test_hello above must begin with "test".

MethodsMethods

run_testsrun_tests

run_tests: Runs all tests. Normally used on talk page of unit tests.

    {{#invoke:Bananas/testcases|run_tests}}

If differs_at=1 is specified, a column will be added showing the first character position where the expected and actual results differ.

    {{#invoke:Bananas/testcases|run_tests|differs_at=1}}

The test methods below that also accept an optional options parameter which can give processing instructions for the results. If this parameter is used, it must be a Lua table. The following fields in that table are recognized:

varying=1 : any non-nil and non-false value allows the test to be executed, and both expected and actual values are compared; but if there's a difference, we'll see the warning icon [!] instead of the failure icon [No]: this test is still counted as successful and not as a failure when results are different; when the results are identical, you still see the success icon [OK]. This is useful for example when checking the result of a test whose output is constantly changing (e.g. a function that returns a random value), or for comparing the actual results of a tested module with the results of an legacy template or module implementing a new better behavior that was purposely changed.
nowiki=1 : any non-nil and non-false value allows displaying expected and actual results in plain text form where its content will not be preprocessed by MediaWiki. Useful for tests that generate external links or links to images or videos and other complex formats, or that include named or numeric character entities for characters (such as ' ', ' ', or ' ', or ' ') which could be equivalently encoded using plain UTF-8 (such as ' ' for the non-breaking space NBSP, which may also be encoded in a Lua string constant provided for the expected value by using decimal escapes for each byte of the UTF-8 sequence, such as '\194\160' or other Lua escapes, or directly in plain text if they are not controls, reserved in the Lua syntax for string delimiters) with the same rendering in HTML. It also allows defining the expected value string more easily for designing tests providing an explicit one.
Note that strings compared in Lua are independant of the HTML or MediaWiki parsing and independant of the UTF-8 encoding implicitly used by MediaWiki, so named or numeric character entities are considered to be distinct from plain-text encoding; but the plain-text UTF-8 encoding of string constants (when it is permitted by the syntax) is equivalent to encodings using Lua escapes (starting by \).
htmlize=1 : any non-nil and non-false value is similar to nowiki=1 but allows expected and actual results in plain wikitext form, where its content will not be preprocessed by MediaWiki, but where also the HTML tags and attributes, HTML comments, and other MediaWiki magic keywords or the MediaWiki wiki syntax for tables, lists, and images will not be postprocessed by Mediawiki, showing the wikitext exactly as it is generated. In addition, the ampersand character & (also used in HTML characters entities) will be rendered as &, and plaintext whitespaces will be rendered in the displayed plaintext as numeric HTML character entities (	 for TAB, 
 for NEWLINE,   for the ASCII SPACE,   for NBSP) so that different encodings in the generated output can be easily distringuished visually and that the Mediawiki postprocessing (from Wikitext to HTML) will not strip or compress these whitespaces, and will not reorder them outside embedded HTML elements (when tidying its HTML output).
asciionly=1 : any non-nil and non-false value allows displaying expected and actual results with all non-ASCII characters replaced by their UTF-8 encoding rendered byte per byte in decimal with Lua escapes (like \194\160 for a plain-text non-breaking space). This does not transform HTML escapes, but this can be combined with nowiki=1 or htmlize=1. Useful for debugging purpose when invalid UTF-8 sequences are generated by the code to test, or to help locating in the results invisible controls and or valid Unicode codepoints forbidden in HTML, all of these would not be displayed correctly in the results.
some test methods below accept additional options.

headingheading

self:heading(text): generates an additional row in the results table, displaying some arbitrary text spanning all columns of the table. The text may contain wiki markup. Can be useful to describe the test of the meaning of "Actual" and "Expected" columns in the generated results table. Can also be used to describe a subgroup of individual tests executed and rendered in the following rows of results.

preprocess_equalspreprocess_equals

self:preprocess_equals(text, expected, options): Gives a piece of wikitext to preprocess and an expected resulting value. Scripts and templates can be invoked in the same manner they would be in a page.

    self:preprocess_equals('{{#invoke:Bananas | hello}}', 'Hello, world!', {nowiki=1})

preprocess_equals_manypreprocess_equals_many

preprocess_equals_many(prefix, suffix, cases, options): Performs a series of preprocess_equals() calls on a set of given pairs. Automatically adds the given prefix and suffix to each text.

    self:preprocess_equals_many(
        '{{#invoke:BananasArgs|add|', '}}',
        {
            { '2|3', '5' },
            { '-2|2', '0' },
        }
    )
    self:preprocess_equals_many(
        '{{#invoke:Coordinates/sandbox|externalLink|site=', '}}',
        {
            { 'GoogleMaps|globe=Mars|lat=-14.6|lon=175.5',
              '//www.google.com/mars/#lat=-14.6&lon=175.5&zoom=8'
            },
            { 'GeoHack|globe=Moon|lat=0.655930|lon=23.470173|lang=en',
              '//geohack.toolforge.org/geohack.php?pagename=Module_talk:Coordinates/sandbox/testcases&params=0.655930_N_23.470173_E_globe:Moon_&language=en'
            },
        }
    )
    self:preprocess_equals_many(
        '{{#invoke:Coordinates/sandbox|GeoHack_link|lat=51.48|lon=0|lang=', '}}',
        {
            { 'en',
              '<span class="plainlinksneverexpand">[//geohack.toolforge.org/geohack.php?pagename=Module_talk:Coordinates/sandbox/testcases&params=51.48_N_0_E_globe:Earth_&language=en 51°&nbsp;28′&nbsp;48″&nbsp;N, 0°&nbsp;00′&nbsp;00″&nbsp;E]</span>'
            },
            { 'ru',
              '<span class="plainlinksneverexpand">[//geohack.toolforge.org/geohack.php?pagename=Module_talk:Coordinates/sandbox/testcases&params=51.48_N_0_E_globe:Earth_&language=ru 51°&nbsp;28′&nbsp;48″&nbsp;с.&nbsp;ш., 0°&nbsp;00′&nbsp;00″&nbsp;в.&nbsp;д.]</span>'
            },
        },
        -- nowiki disables the MediaWiki expansion of templates, so that results also display the generated wikitexts (including wikilinks, categories, wikitables,
        -- wikilists, HTML tags and attributes, magic keywords), without postprocessing them to HTML (such as transforming external links with an additional icons)
        { nowiki = 1 }
    )

preprocess_equals_many_samepreprocess_equals_many_same

preprocess_equals_many_same(prefix, suffix, cases, expected, options): Performs a series of preprocess_equals() calls on a set of input cases with the same expected output. Automatically adds the given prefix and suffix to each text.

    self:preprocess_equals_many(
        '{{#invoke:BananasArgs|add|', '}}',
        {
            '2|3',
            '3|2',
            '10|-5',
        },
        '5',
        -- options here are not necessary, given the format of the expected output above
        { nowiki = 1 }
    )

preprocess_equals_preprocesspreprocess_equals_preprocess

self:preprocess_equals_preprocess(text, expected, options): Gives two pieces of wikitext to preprocess and determines if they produce the same value. Useful for comparing scripts to existing templates.

    self:preprocess_equals_preprocess(
        '{{#invoke:Bananas | hello}}',
        '{{Hello}}',
         -- nowiki doesn't prevent the template expansion of '{{Hello}}' by MediaWiki, as the expected and actual results are preprocessed,
         -- but it shows the wikitext with tags and attributes generated in the MediaWiki syntax, without rendering it as normal HTML.
        { nowiki = 1 }
    )

preprocess_equals_preprocess_manypreprocess_equals_preprocess_many

self:preprocess_equals_preprocess_many(prefix, suffix, cases, options): Performs a series of preprocess_equals_preprocess() calls on a set of given pairs. The prefix/suffix supplied for both arguments is added automatically. If in any case the second part is not specified, the first part will be used.

    self:preprocess_equals_preprocess_many(
        '{{#invoke:Foo | spellnum |', '}}',
        '{{spellnum', '}}',
        {
          { '2' }, -- equivalent to {'2','2'},
          { '-2', '-2.0' },
        },
        -- options here are not necessary, given the format of the expected outputs above
        { nowiki = 1 }
    )

equalsequals

self:equals(name, actual, expected, options): Gives a computed value and the expected value, and checks if they are equal according to the == operator. Useful for testing modules that are designed to be used by other modules rather than using #invoke.

This is intended to perform tests using calls internal in Lua, without preprocessing the input text for the actual value as if it was a MediaWiki syntax. The test actual value provided as a Lua expression.
Now equals() allows comparing results independantly of their type, and even allows checking circular references in tables and to check them.: In such case, special reference values are inserted, containing the datatype name, an hash sign and an ordinal id (such as :table#2: for a reference to the 2nd table in the result, tables basing counted from their opening [ character in the result).; Metatables attached to tables (if they are set) may also be dumped and compared using an empty key []. For that you must set the option include_mt to include metatables in the expected and actual results.; It works even when datatypes are different, between numbers, booleans, strings, tables, functions or nil.; It also works when keys have different types, or there are integer keys out of sequence; keys in tables are sorted in a stable order (starting by integer keys in sequence 1..N, then other integers, booleans, strings, tables, references to functions, other references).

    self:equals('Simple addition', 2 + 2, 4)
    self:equals('Simple equality test', 2 == 2, true)
    self:equals('Test returning tables',
        {{2 == 2}},
        {{true}},
        -- The nowiki option avoids MediaWiki postprocessing of Lua tables as if it was a Mediawiki syntax to expand a template.
        {nowiki=1}
    )
    self:equals('Test returning HTML',
        mw.html.create('span'):css('display', 'none'):wikitext('dummy'):tostring(),
        '<span style="display:none">dummy</span>',
        -- The nowiki option avoids MediaWiki postprocessing to HTML (that would be invisible in the expected and actual values shown in the results table)
        {nowiki=1}
    )

equals_deepequals_deep

self:equals_deep(name, actual, expected, options): Like equals, but handles tables by doing a deep comparison. Neither value should contain circular references, as they are not handled by the current implementation and may result in an infinite loop.

Legacy, now fully equivalent to equals(). The old restriction of use shown above no longer applies and you can compare Lua values with any type, including tables with circular references either in their keys, mapped values, or assigned metatables.

    self:equals_deep('Table comparison', createRange(1,3), {1,2,3}, {nowiki=1}) -- legacy
    self:equals('Table comparison', createRange(1,3), {1,2,3}, {nowiki=1}) -- now equivalent

Code

-- UnitTester provides unit testing for other Lua scripts. For details see [[Wikipedia:Lua#Unit_testing]].
-- For user documentation see talk page.
local UnitTester = {}

local libraryUtil = require 'libraryUtil'
local checkType, checkTypeMulti = libraryUtil.checkType, libraryUtil.checkTypeMulti

--------------------------------------------------------------------------------------------------------------------------

local val_to_str; do
	-- Cached function references (for performance).
	local byte   = string.byte
	local find   = string.find
	local match  = string.match
	local gsub   = string.gsub
	local format = string.format
	local insert = table.insert
	local sort   = table.sort
	local concat = table.concat
	-- For escaping string values
	local str_escape_map = {
		['\a'] = '\\a', ['\b'] = '\\b', ['\t'] = '\\t', ['\n'] = '\\n',
		['\v'] = '\\v', ['\f'] = '\\f', ['\r'] = '\\r', ['\\'] = '\\\\' }
	local str_escape_replace = function(c)
		return str_escape_map[c] or format('\\%03d', byte(c))
	end
	-- Keys are comparable only if the same type, otherwise just sort them by type.
	local types_order, ref_types_order = {
		['number'] = 0, ['boolean'] = 1, ['string'] = 2, ['table'] = 3,
		['function'] = 4 }, 5
	function compare_keys(k1, k2)
		local t1, t2 = type(k1), type(k2)
		if t1 ~= t2 then -- not the same type
		   return (types_order[t1] or ref_types_order)
				< (types_order[t2] or ref_types_order)
		elseif t1 == 'number' or t1 == 'string' then -- comparing numbers (including NaNs or infinites) or strings
		   return k1 < k2 -- keys with the same comparable type
		elseif t1 == 'boolean' then -- comparing booleans
		   return not k1 -- sort false before true
		else -- comparing references
		   return tostring(k1) < tostring(k2)
		end
	end
	-- String keys matching valid identifiers that are reserved by Lua.
	local reserved_keys = {
		['and']      = 1, ['break'] = 1, ['do']    = 1, ['else']   = 1,
		['elseif']   = 1, ['end']   = 1, ['false'] = 1, ['for']    = 1,
		['function'] = 1, ['if']    = 1, ['in']    = 1, ['local']  = 1,
		['nil']      = 1, ['not']   = 1, ['or']    = 1, ['repeat'] = 1,
		['return']   = 1, ['then']  = 1, ['true']  = 1, ['until']  = 1,
		['while']    = 1 }
	-- Main function.
	val_to_str = function(val, options)
		-- Decode and cache the options.
		local include_mt  = options and options.include_mt
		local prettyprint = options and options.prettyprint
		local asciionly   = options and options.asciionly
		-- Precompute the output formats depending on options.
		local open   = prettyprint and '{ '  or '{'
		local equals = prettyprint and ' = ' or '='
		local comma  = prettyprint and ', '  or ','
		local close  = prettyprint and ' }'  or '}'
		-- What to escape: C0 controls, the backslash, and optionally non-ASCII bytes.
		local str_escape_pattern = asciionly and '[%z\001-\031\\\127-\255]' or '[%z\001-\031\\\127]'
		 -- Indexed references (mapped to ids), and counters per ref type.
		local ref_ids, ref_counts = {}, {}
		-- Helper needed to detect recursive tables and avoid infinite loops.
		local function visit(ref)
			local typ = type(ref)
			if typ == 'number' or typ == 'boolean' then
				return tostring(ref)
			elseif typ == 'string' then
				if find(ref, "'") then
				   str_escape_map['"'] = '\\"'
				   return '"' .. gsub(ref, str_escape_pattern, str_escape_replace) .. '"'
				else
				   str_escape_map['"'] = '"'
				   return "'" .. gsub(ref, str_escape_pattern, str_escape_replace) .. "'"
				end
			elseif typ == 'table' then
				local id = ref_ids[ref]
				if id then
					return ':' .. typ .. '#' .. id .. ':'
				end
				id = (ref_counts[typ] or 0) + 1; ref_ids[ref], ref_counts[typ] = id, id
				-- First dump keys that are in sequence.
				local result, sequenced, keys = {}, {}, {}
				for i, val in ipairs(ref) do
					insert(result, visit(val))
					sequenced[i] = true
				end
				-- Then dump other keys out of sequence, in a stable order.
				for key, _ in pairs(ref) do
					if not sequenced[key] then
						insert(keys, key)
					end
				end
				sequenced = nil -- Free the temp table no longer needed.
				-- Sorting keys (of any type) is needed for stable comparison of results.
				sort(keys, compare_keys)
				for _, key in ipairs(keys) do
					insert(result,
						(type(key) == 'string' and
							not reserved_keys[key] and match(key, '^[%a_][%d%a_]*$') and
							key or '[' .. visit(key) .. ']') ..
						equals .. visit(ref[key]))
				end
				keys = nil -- Free the temp table no longer needed.
				-- Finally dump the metatable (with pseudo-key '[]'), if there's one.
				if include_mt then
					ref = getmetatable(ref)
					if ref then
						insert(result, '[]' .. equals .. visit(ref))
					end
				end
				-- Pack the result string.
				-- TODO: improve pretty-printing with newlines/indentation
				return open .. concat(result, comma) .. close
			elseif typ ~= 'nil' then -- other reference types (function, userdata, etc.)
				local id = ref_ids[ref]
				if not id then
					id = (ref_counts[typ] or 0) + 1; ref_ids[ref], ref_counts[ref] = id, id
				end
				return ':' .. typ .. '#' .. id .. ':'
			else
				return 'nil'
			end
		end
		return visit(val)
	end
end

--------------------------------------------------------------------------------------------------------------------------

local htmlize; do -- For rendering valid UTF-8 HTML code (possibly multiline), as a visible plain text (on a single line that can fit in a wikitable cell)
	local escaping_ascii = '[\t\n\r&<>%[%]_{|}~]' -- ASCII characters encoded on 1 byte in UTF-8, that should be displayed as HTML entities below.
	local html_entities = { -- All named character entities should be valid in HTML 5.2+ (https://html.spec.whatwg.org/multipage/named-characters.html).
		['\t'] = '&#9;', -- Display whitespace controls visibly on one-line plaintext
		['\n'] = '&#10;',
		['\r'] = '&#13;',
		['&'] = '&amp;', -- Required here, because we use '&' for rendering all character entities in this table.
		['\194\160'] = '&nbsp;', -- U+00A0 (NON-BREAKING SPACE, NBSP): code point value = 160 (UTF-8: 0xC2 0xA0).
		['\194\173'] = '&shy;', -- U+00AD (SOFT HYPHEN, SHY): code point value = 173 (UTF-8: 0xC2 0xAD).
		['\226\128\128'] = '&#x2000;', -- U+2000 (EN QUAD): code point value = 8192 (UTF-8: 0xE2 0x80 0x80).
		['\226\128\129'] = '&#x2001;', -- U+2001 (EM QUAD): code point value = 8193 (UTF-8: 0xE2 0x80 0x81).
		['\226\128\130'] = '&ensp;', -- U+2002 (EN SPACE): code point value = 8194 (UTF-8: 0xE2 0x80 0x82).
		['\226\128\131'] = '&emsp;', -- U+2003 (EM SPACE): code point value = 8195 (UTF-8: 0xE2 0x80 0x83).
		['\226\128\132'] = '&emsp13;', -- U+2004 (THREE-PER-EM SPACE): code point value = 8196 (UTF-8: 0xE2 0x80 0x84).
		['\226\128\133'] = '&emsp14;', -- U+2005 (FOUR-PER-EM SPACE): code point value = 8197 (UTF-8: 0xE2 0x80 0x85).
		['\226\128\134'] = '&#x2006;', -- U+2006 (SIX-PER-EM SPACE): code point value = 8198 (UTF-8: 0xE2 0x80 0x86).
		['\226\128\135'] = '&numsp;', -- U+2007 (FIGURE SPACE, TABULAR SPACE): code point value = 8199 (UTF-8: 0xE2 0x80 0x87).
		['\226\128\136'] = '&puncsp;', -- U+2008 (PUNCTUATION SPACE): code point value = 8200 (UTF-8: 0xE2 0x80 0x88).
		['\226\128\137'] = '&thinsp;', -- U+2009 (THIN SPACE): code point value = 8201 (UTF-8: 0xE2 0x80 0x89).
		['\226\128\138'] = '&hairsp;', -- U+200A (HAIR SPACE): code point value = 8202 (UTF-8: 0xE2 0x80 0x8A).
		['\226\128\139'] = '&ZeroWidthSpace;', -- U+200B (ZERO-WIDTH SPACE, ZWSP): code point value = 8203 (UTF-8: 0xE2 0x80 0x8B).
		['\226\128\140'] = '&zwnj;', -- U+200C (ZERO-WIDTH NON-JOINER, ZWNJ): code point value = 8204 (UTF-8: 0xE2 0x80 0x8C).
		['\226\128\141'] = '&zwj;', -- U+200D (ZERO-WIDTH JOINER, ZWJ): code point value = 8205 (UTF-8: 0xE2 0x80 0x8D).
		['\226\128\142'] = '&lrm;', -- U+200E (LEFT-TO-RIGHT MARK, LRM): code point value = 8206 (UTF-8: 0xE2 0x80 0x8E).
		['\226\128\143'] = '&rlm;', -- U+200F (RIGHT-TO-LEFT MARK, RLM): code point value = 8207 (UTF-8: 0xE2 0x80 0x8F).
		['\226\128\168'] = '&#x2028;', -- U+2028 (LINE SEPARATOR, LSEP): code point value = 8232 (UTF-8: 0xE2 0x80 0xA8).
		['\226\128\169'] = '&#x2029;', -- U+2029 (PARAGRAPH SEPARATOR, PSEP): code point value = 8233 (UTF-8: 0xE2 0x80 0xA9).
		['\226\128\170'] = '&#x202A', -- U+202A (LEFT-TO-RIGHT EMBEDDING, LRE): code point value = 8234 (UTF-8: 0xE2 0x80 0xAA).
		['\226\128\171'] = '&#x202B;', -- U+202B (RIGHT-TO-LEFT EMBEDDING, RLE): code point value = 8235 (UTF-8: 0xE2 0x80 0xAB).
		['\226\128\172'] = '&#x202C;', -- U+202C (POP DIRECTIONAL FORMATTING, PDF): code point value = 8236 (UTF-8: 0xE2 0x80 0xAC).
		['\226\128\173'] = '&#x202D;', -- U+202D (LEFT-TO-RIGHT OVERRIDE, LRO): code point value = 8237 (UTF-8: 0xE2 0x80 0xAD).
		['\226\128\174'] = '&#x202E;', -- U+202E (RIGHT-TO-LEFT OVERRIDE, RLO): code point value = 8238 (UTF-8: 0xE2 0x80 0xAE).
		['\226\128\175'] = '&#x202F;', -- U+202F (NARROW NON-BREAKING SPACE, NNBSP): code point value = 8239 (UTF-8: 0xE2 0x80 0xAF).
		['\226\129\159'] = '&MediumSpace;', -- U+205F (MEDIUM MATHEMATICAL SPACE, MMSP): code point value = 8239 (UTF-8: 0xE2 0x81 0x9F).
		['\226\129\160'] = '&#x2060;', -- U+2060 (WORD JOINER, WJ): code point value = 8288 (UTF-8: 0xE2 0x81 0xA0).
		['\226\129\161'] = '&#x2061;', -- U+2061 (FUNCTION APPLICATION, FA): code point value = 8289 (UTF-8: 0xE2 0x81 0xA1).
		['\226\129\162'] = '&#x2062;', -- U+2062 (INVISIBLE TIMES): code point value = 8290 (UTF-8: 0xE2 0x81 0xA2).
		['\226\129\163'] = '&#x2063;', -- U+2063 (INVISIBLE SEPARATOR): code point value = 8291 (UTF-8: 0xE2 0x81 0xA3).
		['\226\129\164'] = '&#x2064;', -- U+2064 (INVISIBLE PLUS): code point value = 8292 (UTF-8: 0xE2 0x81 0xA4).
		['\226\129\166'] = '&#x2066;', -- U+2066 (LEFT-TO-RIGHT ISOLATE, LRI): code point value = 8294 (UTF-8: 0xE2 0x81 0xA6).
		['\226\129\167'] = '&#x2067;', -- U+2067 (RIGHT-TO-LEFT ISOLATE, RLI): code point value = 8295 (UTF-8: 0xE2 0x81 0xA7).
		['\226\129\168'] = '&#x2068;', -- U+2068 (FIRST STRONG ISOLATE, FSI): code point value = 8296 (UTF-8: 0xE2 0x81 0xA8).
		['\226\129\169'] = '&#x2069;', -- U+2069 (POP DIRECTIONAL ISOLATE, PDI): code point value = 8297 (UTF-8: 0xE2 0x81 0xA9).
		['\227\128\128'] = '&#x3000;', -- U+3000 (IDEOGRAPHIC SPACE): code point value = 12288 (UTF-8: 0xE3 0x80 0x80).
		['\239\187\191'] = '&#xFEFF;', -- U+FEFF (ZERO-WIDTH NON-BREAKING SPACE, ZWNSP, BYTE ORDER MARK, BOM): code point value = 65279 (UTF-8: 0xEF 0xBB 0xBF).
		['\239\191\188'] = '&#xFFFC;', -- U+FFFC (OBJECT REPLACEMENT CHARACTER, ORC): code point value = 65532 (UTF-8: 0xEF 0xBF 0xBC).
		['\239\191\189'] = '&#xFFFD;', -- U+FFFD (REPLACEMENT CHARACTER, RC): code point value = 65532 (UTF-8: 0xE2 0x80 0xA8).
	}
	local U_FFFD = '\239\191\189' -- U+FFFD (REPLACEMENT CHARACTER)
	local insert, concat = table.insert, table.concat
	local function dump(s) -- For dumping invalid bytes in hexadecimal after U+FFFD (with options.invalid = 3).
		local t = {}
		for i = 1, #s do
			insert(t, ('%02X'):format(s:byte(i)))
		end
		return U_FFFD .. concat(t) .. ';'
	end
	local function many(s) -- For replacing each invalid byte by U+FFFD (with options.invalid = 2).
		return U_FFFD:rep(#s)
	end
	local forbidden = '[%z\001-\007\011\012\014-\031\127-\255]+' -- ASCII controls forbidden in HTML, and non-ASCII bytes.
	htmlize = function(text, options)
		local asciionly = options and options.asciionly -- Encode valid non-ASCII characters using multiple bytes in UTF-8 as HTML entities.
		local replaceby = options and options.invalid and -- How to replace a sequence of bytes that are invalid in UTF-8 or forbidden in HTML:
			(	options.invalid == 0 and ''     -- either discard the sequence silently (length minimized).
			or	options.invalid == 1 and U_FFFD -- or replace all bytes in the sequence by a single U+FFFD (length reduced)
			or	options.invalid == 2 and many   -- or replace each byte in the sequence by U+FFFD (length preserved),
			or	options.invalid == 3 and dump   -- or replace by U+FFFD + hexadecimal dump (length increased),
			) or dump -- default replacement
		return tostring(text)
			:gsub(-- Split the text in pairs of (ASCII or leading or invalid bytes, trailing bytes) and filter them.
				'([%z\001-\127\192-\255]*)([\128-\191]*)',
				function(s, t)
					local a = s:byte(-1) -- We just need to test the last leading byte before any trailing bytes.
					if not(a) or a < 194 or a > 244 then -- The last leading byte is missing, ASCII or invalid in UTF-8.
						-- All trailing bytes after a in t are also invalid.
						return (s .. t):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities)
					elseif a < 224 then -- The last valid leading byte should be followed only by 1 valid trailing byte.
						-- The last non-ASCII character u (if it's valid) is encoded in UTF-8 as 2 bytes: a, b.
						local u, b = s:sub(-1) .. t:sub(1, 1), t:byte(1)
						return s:sub(1, -2):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities) ..
							(	(b	and b > (a > 194 and 127 or 159) and b < 192
									and (html_entities[u]
										or asciionly and ('&#x%X;'):format((a - 192) * 64 + b - 128)
										or u)
								or ''
								) .. t:sub(2):gsub(forbidden, replaceby) -- All other trailing bytes after b in t are also invalid.
							or	(s:sub(-1) .. t):gsub(forbidden, replaceby))
					elseif a < 240 then -- The last valid leading byte should be followed only by 2 valid trailing bytes.
						-- The last non-ASCII character u (if it's valid) is encoded in UTF-8 as 3 bytes: a, b, c.
						local u, b, c = s:sub(-1) .. t:sub(1, 2), t:byte(1), t:byte(2)
						return s:sub(1, -2):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities) ..
							(	(c	and c > 127 and c < 192 and b > 127 and b < 192
									and (html_entities[u]
										or asciionly and ('&#x%X;'):format(((a - 224) * 64 + b - 128) * 64 + c - 128)
										or u)
								or ''
								) .. t:sub(3):gsub(forbidden, replaceby) -- All other trailing bytes after d in t are also invalid.
							or	(s:sub(-1) .. t):gsub(forbidden, replaceby))
					elseif a < 245 then -- The last valid leading byte should be followed only by 3 valid trailing bytes.
						-- The last non-ASCII character u (if it's valid) is encoded in UTF-8 as 4 bytes: a, b, c, d.
						local u, b, c, d = s:sub(-1) .. t:sub(1, 3), t:byte(1), t:byte(2), t:byte(3)
						return s:sub(1, -2):gsub(forbidden, replaceby):gsub(escaping_ascii, html_entities) ..
							(	(d	and d > 127 and d < 192 and c > 127 and c < 192 and b > (a < 244 and 127 or 143) and b < 192
									and (html_entities[u]
										or asciionly and ('&#x%X;'):format((((a - 240) * 64 + b - 128) * 64 + c - 128) * 64 + d - 128)
										or u)
								or ''
								) .. t:sub(4):gsub(forbidden, replaceby) -- All other trailing bytes after d in t are also invalid.
							or	(s:sub(-1) .. t):gsub(forbidden, replaceby))
					end
				end
			)
			:gsub('^ ', '&#32;') -- Avoids the compression of a leading SPACE character and make it visible.
			:gsub('  ', ' &#32;') -- Avoids the compression of repeated SPACE characters and make them visible in pairs.
			:gsub(' $', '&#32;') -- Avoids the compression of a trailing SPACE character and make it visible.
			or nil -- Needed in Lua to discard the additional count of substitutions returned by a trailing call to gsub().
	end
end

--------------------------------------------------------------------------------------------------------------------------

local function first_difference(a, b, options)
	checkType('UnitTester:first_difference', 3, options, 'table', true)
	if a == b then
		return ''
	elseif type(a) ~= type(b) then
		return ('%s ≠ %s'):format(type(a), type(b))
	elseif type(a) == 'string' then
		local i, c, d, e = 1
		while true do
			c, d = a:byte(i) or -1, b:byte(i) or -1
			e = c < d and d or c
			if c ~= d or
				e >= 192 and a:byte(i + 1) ~= b:byte(i + 1) or
				e >= 224 and a:byte(i + 2) ~= b:byte(i + 2) or
				e >= 240 and a:byte(i + 3) ~= b:byte(i + 3) then
				return ('%d: %s%s ≠ %s%s'):format(i,
					mw.text.nowiki(htmlize(val_to_str(a:sub(i, (c >= 240 and 3 or c >= 224 and 2 or c >= 192 and 1 or 0) + i), options))),
					c >= 240 and ' (4 bytes)' or c >= 224 and ' (3 bytes)' or c >= 192 and ' (2 bytes)' or '',
					mw.text.nowiki(htmlize(val_to_str(b:sub(i, (d >= 240 and 3 or d >= 224 and 2 or d >= 192 and 1 or 0) + i), options))),
					d >= 240 and ' (4 bytes)' or d >= 224 and ' (3 bytes)' or d >= 192 and ' (2 bytes)' or '')
			end
			i = (c >= 240 and 4 or c >= 224 and 3 or c >= 192 and 2 or 1) + i
		end
	elseif type(a) == 'table' then
		local m = #a < #b and #a or #b
		for i = 1, m do
			if a[i] ~= b[i] then
				return ('%i: %s ≠ %s'):format(i, mw.text.nowiki(htmlize(val_to_str(a[i]), options)), mw.text.nowiki(htmlize(val_to_str(b[i]), options)))
			end
		end
	else
		return ('%s ≠ %s'):format(htmlize(val_to_str(a)), htmlize(val_to_str(b)))
	end
end

--------------------------------------------------------------------------------------------------------------------------

local result_table; do
	local format = string.format
	result_table = {}
	local meta = {
		insert = function(self, ...)
				local n = #self
				for i = 1, select('#', ...) do
					local val = select(i, ...)
					if val ~= nil then
						n = n + 1
						self[n] = tostring(val)
					end
				end
			end,
		insert_format = function(self, ...)
				self:insert(format(...))
			end,
		concat = table.concat,
		tostring = table.concat,
	}
	meta.__index = meta
	setmetatable(result_table, meta)
end

local function return_varargs(...)
	return ...
end

--------------------------------------------------------------------------------------------

function UnitTester:heading(text)
	checkType('UnitTester:heading', 1, text, 'string', false)
	result_table:insert(
		'|-\n!scope="colgroup" colspan="' ..
		tostring(self.columns) ..
		'" style="background:#FFD;color:#000;font-weight:normal;text-align:left"|' ..
		text ..
		'\n')
end

--------------------------------------------------------------------------------------------
-- All "preprocess" tests require that each case returns a single string.
-- As these tests are calling the Mediawiki preprocessor, they are much slower, use more resources, and may
-- fail with MediaWiki timeout errors (displaying no result at all), so they can't be too much comprehensive.

function UnitTester:preprocess_equals(text, expected, options)
	checkType('UnitTester:preprocess_equals', 1, self, 'table', false)
	checkType('UnitTester:preprocess_equals', 2, text, 'string', false)
	checkType('UnitTester:preprocess_equals', 3, expected, 'string', false)
	checkType('UnitTester:preprocess_equals', 4, options, 'table', true)
	local actual = self.frame:preprocess(text)
	local varying = options and options.varying
	local display =
		(options and options.display) and options.display or
		(options and options.htmlize) and htmlize or
		(options and options.nowiki) and mw.text.nowiki or
		return_varargs
	result_table:insert(
		'|-\n||' ..
		self['icon' ..
			(actual == expected and
				'Tick' or
			varying and
				'Warn' or
				'Cross'
			)
		] ..
		'||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		mw.text.nowiki(text) ..
		'</bdi>||<bdi style="' ..
		(type(expected) ~= 'string' and
			'background:#FCC;' or
			''
		) ..
		'border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		(type(expected) ~= 'string' and
			mw.text.nowiki(val_to_str(expected)) or
			display(expected)
		) ..
		'</bdi>||<bdi style="' ..
		(type(actual) ~= 'string' and 'background:#FCC;' or '') ..
		'border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		(type(actual) ~= 'string' and
			mw.text.nowiki(val_to_str(actual)) or
			display(actual)
		) ..
		'</bdi>' ..
		(self.differs_at and
			'||' .. (	(type(expected) ~= 'string' or type(actual) ~= 'string') and self.iconCross
					or	first_difference(expected, actual, options)
					)
		or	''
		) ..
		'\n')
	if type(expected) ~= 'string' or
		type(actual) ~= 'string' or
		not varying and actual ~= expected then
		self.num_failures = self.num_failures + 1
	end
end

function UnitTester:preprocess_equals_preprocess(actual, expected, options, text)
	checkType('UnitTester:preprocess_equals_preprocess', 1, self, 'table', false)
	checkType('UnitTester:preprocess_equals_preprocess', 2, actual, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess', 3, expected, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess', 4, options, 'table', true)
	checkType('UnitTester:preprocess_equals_preprocess', 4, text, 'string', true)
	local text = text or actual
	local errs; do
		-- Protected call to the preprocessor which may fail: detect and preserve errors.
		local s1, s2
		s1, actual = pcall(self.frame.preprocess, self.frame, actual)
		if not s1 then
			actual = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
				val_to_str(s1) .. ' --[=[#ERROR! ' ..
				(type(actual) == 'string' and actual or val_to_str(actual)) ..
				']=]</bdi>'
		end
		s2, expected = pcall(self.frame.preprocess, self.frame, expected)
		if not s2 then
			expected = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
				val_to_str(s2) .. ' --[=[#ERROR! ' ..
				(type(expected) == 'string' and expected or val_to_str(expected)) ..
				']=]</bdi>'
		end
		-- If there was no processing error, check the return types (should be strings).
		if not (s1 and s2) then
			if type(actual) ~= 'string' then
				actual = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
					val_to_str(actual) .. '</bdi>'
			end
			if type(expected) ~= 'string' then
				expected = '<bdi style="background:#FCC;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
					val_to_str(expected) .. '</bdi>'
			end
			errs = true
		end
	end
	if errs then
		result_table:insert(
			'|-\n||' ..
			self.iconCross ..
			'||<bdi style="border:1px solid #EAECF0;padding:1px;background:#F8F9FA;white-space:pre-wrap">' ..
			mw.text.nowiki(text) ..
			'</bdi>||<bdi style="background:#' ..
			(type(expected) ~= 'string' and 'F8F9FA' or 'FFF') ..
			';border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
			(type(expected) ~= 'string' and mw.text.nowiki(expected) or display(expected)) ..
			'</bdi>||<bdi style="background:#' ..
			(type(actual) ~= 'string' and 'F8F9FA' or 'FFF') ..
			'border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
			(type(actual) ~= 'string' and mw.text.nowiki(actual) or display(actual))..
			'</bdi>' ..
			(self.differs_at and
				'||' ..
				self.iconCross
				or ''
			) ..
			'\n')
		self.num_failures = self.num_failures + 1
		return
	end
	if options and options.stripmarker == true then
		-- Option to ignore ANY strip marker when comparing actual to expected.
		local _, stripmarker_id = expected:match('(\127[^\127]*UNIQ%-%-%l+%-)(%x+)(%-QINU[^\127]*\127)')
		if stripmarker_id then
			actual = actual:gsub(pattern, '%1' .. stripmarker_id .. '%3')
		end
	elseif options and options.templatestyles == true then
		-- When module rendering has templatestyles strip markers, use ID from expected to prevent false test fail.
		-- Get the strip marker id for templatestyles from expected (the reference); ignore first capture in pattern.
		-- Strip marker pattern for '<templatestyles src="..." />' .
		local _, stripmarker_id = expected:match('(\127[^\127]*UNIQ%-%-templatestyles%-)(%x+)(%-QINU[^\127]*\127)')
		if stripmarker_id then
			actual = actual:gsub(pattern, '%1' .. stripmarker_id .. '%3') -- Replace actual id with expected id; ignore second capture in pattern.
		end
	end
	local varying = options and options.varying
	local display =
		(options and options.display) and options.display or
		(options and options.htmlize) and htmlize or
		(options and options.nowiki) and mw.text.nowiki or
		return_varargs
	result_table:insert(
		'|-\n||' ..
		self['icon' ..
			(actual == expected and
				'Tick' or
			varying and
				'Warn' or
				'Cross'
			)
		] ..
		'||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		mw.text.nowiki(text) ..
		'</bdi>||<bdi style="background:#FFF;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		display(expected) ..
		'</bdi>||<bdi style="background:#FFF;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		display(actual) ..
		(self.differs_at and
			'||' .. first_difference(expected, actual, options)
		or	''
		) ..
		'</bdi>' ..
		'\n'
	)
	if not varying and actual ~= expected then
		self.num_failures = self.num_failures + 1
	end
end

function UnitTester:preprocess_equals_many(prefix, suffix, cases, options, text)
	checkType('UnitTester:preprocess_equals_many', 1, self, 'table', false)
	checkType('UnitTester:preprocess_equals_many', 2, prefix, 'string', false)
	checkType('UnitTester:preprocess_equals_many', 3, suffix, 'string', false)
	checkType('UnitTester:preprocess_equals_many', 4, cases, 'table', false)
	checkType('UnitTester:preprocess_equals_many', 5, options, 'table', true)
	checkType('UnitTester:preprocess_equals_many', 6, text, 'string', true)
	for _, case in ipairs(cases) do
		self:preprocess_equals(prefix .. case[1] .. suffix, case[2], options, text)
	end
end

function UnitTester:preprocess_equals_many_same(prefix, suffix, cases, expected, options, text)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 1, self, 'table', false)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 2, prefix, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 3, suffix, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 4, cases, 'table', false)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 5, expected, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 6, options, 'table', true)
	checkType('UnitTester:preprocess_equals_preprocess_many_same', 7, text, 'string', true)
	for _, case in ipairs(cases) do
		self:preprocess_equals(prefix .. case .. suffix, expected, options, text)
	end
end

function UnitTester:preprocess_equals_preprocess_many(prefix1, suffix1, prefix2, suffix2, cases, options, text)
	checkType('UnitTester:preprocess_equals_preprocess_many', 1, self, 'table', false)
	checkType('UnitTester:preprocess_equals_preprocess_many', 2, prefix1, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many', 3, suffix1, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many', 4, prefix2, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many', 5, suffix2, 'string', false)
	checkType('UnitTester:preprocess_equals_preprocess_many', 6, cases, 'table', false)
	checkType('UnitTester:preprocess_equals_preprocess_many', 7, options, 'table', true)
	checkType('UnitTester:preprocess_equals_preprocess_many', 8, text, 'string', true)
	for _, case in ipairs(cases) do
		self:preprocess_equals_preprocess(prefix1 .. case[1] .. suffix1, prefix2 .. (case[2] and case[2] or case[1]) .. suffix2, options, text)
	end
end

--------------------------------------------------------------------------------------------
-- All tests without "preprocess" allow each case to return any Lua type for actual and expected.
-- These tests use only Lua functions, are much faster, use less resources.

function UnitTester:equals(text, actual, expected, options)
	checkType('UnitTester:equals', 1, self, 'table', false)
	checkType('UnitTester:equals', 2, text, 'string', false)
	checkType('UnitTester:equals', 5, options, 'table', true)
	expected, actual = val_to_str(expected, options), val_to_str(actual, options)
	local varying = options and options.varying
	local display =
		(options and options.display) and options.display or
		(options and options.htmlize) and htmlize or
		(options and options.nowiki) and mw.text.nowiki or
		return_varargs
	result_table:insert(
		'|-\n||' ..
		self['icon' ..
			(actual == expected and 'Tick'
			or varying and 'Warn'
			or 'Cross'
			)
		] ..
		'||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		mw.text.nowiki(text) ..
		'</bdi>||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		mw.text.nowiki(expected) ..
		'</bdi>||<bdi style="background:#F8F9FA;border:1px solid #EAECF0;padding:1px;white-space:pre-wrap">' ..
		mw.text.nowiki(actual) ..
		'</bdi>' ..
		(self.differs_at and
			'||' .. first_difference(expected, actual, options)
		or	''
		) ..
		'\n'
	)
	if not varying and actual ~= expected then
		self.num_failures = self.num_failures + 1
	end
end
-- Legacy: now UnitTester:equals() is deep by default and properly handles tables
UnitTester.equals_deep = UnitTester.equals

--------------------------------------------------------------------------------------------

function UnitTester:iterate(cases, func)
	checkType('UnitTester:iterate', 1, cases, 'table')
	checkType('UnitTester:iterate', 2, func, 'function')
	func = self[func]
	for i, example in ipairs(cases) do
		checkTypeMulti('UnitTester:iterate(cases)', i, cases, {'table', 'string'})
		if type(example) == 'string' then
			self:heading(example)
		else
			func(self, unpack(example))
		end
	end
end

--------------------------------------------------------------------------------------------

-- Main function that enumerates tests and run them
function UnitTester:run(frame)
	self.frame = frame
	self.options = frame.args.options
	self.differs_at = frame.args.differs_at
	-- Get the list of tests and them into alphabetical order.
	local test_names = {}
	for key, value in pairs(self) do
		if key:find('^test') then
			table.insert(test_names, key)
		end
	end
	table.sort(test_names)
	local thead_rows = 1
	local thead =
			'!scope="col" style="max-width:32%"|Expected\n' ..
			'!scope="col" style="max-width:32%"|Actual\n'
	self.columns = 2
	if self.differs_at then
		thead = thead ..
			'!scope="col" style="width:6em"|Diff. at\n'
		self.columns = self.columns + 1
	end
	thead =
			'|-\n' ..
			'!scope="col"' .. (thead_rows > 1 and ' rowspan="' .. tostring(thead_rows) .. '"' or '') .. ' style="width:32px"|\n' ..
			'!scope="col"' .. (thead_rows > 1 and ' rowspan="' .. tostring(thead_rows) .. '"' or '') .. ' style="max-width:32%"|Text\n' ..
			thead
	self.columns = self.columns + 2
	if not self.iconTick then
		-- Icons are preprocessed early before running tests rather than after packing results.
		-- This reduces the number of template expansions for these icons packed in results.
		self.iconTick = frame:expandTemplate{ title = 'Tick', args = {} }
		self.iconWarn = frame:expandTemplate{ title = 'Warn', args = {} }
		self.iconCross = frame:expandTemplate{ title = 'Cross', args = {} }
	end
	local display_options = self.options and
			'<br /><span style="font-size:smaller;font-weight:normal">' ..
			'Options: <kbd>' .. val_to_str(options) ..
			'</kbd></span>' or ''
	self.num_failures = 0
	-- Add results into the results table.
	for i, test_name in ipairs(test_names) do
		local caption = test_name
			:gsub('^test_?([%d]+[%a]*)_(.-)$', 'Test %1: %2')
			:gsub('^test_?(.-)$', 'Test: %1')
			:gsub('__', ' ')
		result_table:insert(
			'{|class="wikitable" cellspacing="0" cellpadding="0" style="margin:.6em 0 2px;width:100%;max-width:100%;overflow-wrap:anywhere"\n' ..
			'|+|' .. caption .. display_options .. '\n' ..
			thead)

		self[test_name](self)
--[[
		local ok, result = pcall(self[test_name], self)
		if not ok then
			self:heading('<strong>An error occured while running this test:</strong> '.. tostring(result))
		end
--]]

		result_table:insert('|}\n')
	end
	-- Pack results.
	return (self.num_failures == 0 and
		   '<strong style="color:#080">All tests passed.</strong>\n\n' or
		   '<strong style="color:#800">' .. self.num_failures .. ' tests failed.</strong>\n\n'
		) .. frame:preprocess(result_table:concat())
end

function UnitTester:new()
    local o = {}
    setmetatable(o, self)
    self.__index = self
    return o
end

local p = UnitTester:new()
function p.run_tests(frame) return p:run(frame) end

-- Additional exports for tests of this module in the console.
p.val_to_str = val_to_str
p.htmlize = htmlize
p.first_difference = first_difference

return p