Extension talk:DataTable2


On installation difficulties

I added the require_once code to the LocalSettings.php and went to /wiki/mw-config to create the database tables. I got to select my current language and then I got the message:

Fatal error: Class 'DataTable2' not found in /home/storycaf/public_html/wiki/extensions/DataTable2/DataTable2.php

Now for some reason efforts to rewrite LocalSettings.php only end up in a null file, which renders the wiki non-accessible. Hmm. 91.152.253.88 06:47, 28 October 2013 (UTC)

Apparently there's a problem with this in DataTable2.php:
$wgDataTable2 = new DataTable2;
From Special:version:
MediaWiki 1.21.2
PHP 5.3.27 (cgi-fcgi)
MySQL 5.0.96-community-log 91.152.253.88 07:40, 28 October 2013 (UTC)
Thanks for reporting this bug. It is fixed in version 0.8.2 which I have just uploaded. RV1971 (talk) 13:24, 31 October 2013 (UTC)

PostgreSQL supported ?

Please add some information about the supported databases ( PostgreSQL, SQLLite , MySQL ) I have some error messages  :( 89.132.124.186 01:21, 26 December 2013 (UTC)

I'm afraid I don't have an installation with PostgreSQL at hand right now. I use it at home but I need to update my MediaWiki before, which hopefully I'll do in Jan or Feb. In the meantime, could you give me some more details about the error messages? Maybe it's easy to fix. RV1971 (talk) 10:19, 26 December 2013 (UTC)

Column ordering bug and two feature requests

The DataTable2 Extension is a brilliant concept and work RV1971. Your extension has been extremely helpful to me.

Column Ordering - Bug

I have discovered a bug in the data entry <datatable2> tag. After saving the page, the tag shows the content of columns in alphabetical order rather than numerical order. For example, column contents are displayed after saving a page as Column1, Column10, Column11, Column12 ... Column19, Column2, Column20, Column21 ... etc. Thus, this bug only becomes apparent when creating tables with more than 9 columns.

For example, the following code would demonstrate this:

<datatable2 table="column order test" class="wikitable" columns = "column1|column2|column3|column4|column5|column6|column7|column8|column9|column10|column11|column12|column13|column14|column15|column16|column17|column18|column19|column20|column21|column22">
<head>
! column1
! column2
! column3
! column4
! column5
! column6
! column7
! column8
! column9
! column10
! column11
! column12
! column13
! column14
! column15
! column16
! column17
! column18
! column19
! column20
! column21
! column22
</head>
field1|field2|field3|field4|field5|field6|field7|field8|field9|field10|field11|field12|field13|field14|field15|field16|field17|field18|field19|field20|field21|field22
</datatable2>

Fortunately, the <dt2-showtable> tag correctly displays columns as Column1, Column2 ... Column10, Column11, Column12 ... Column19, Column20, Column21 ... etc.

Field Separator - Feature Request No. 1

I have found that the DataTable2 Extension is intolerant of '|' (pipes) field separators with whitespace on either side. I have created a table which has more than twenty columns. Each row of my input data can be quite long and screen wrapping can be an issue. Due to 'human factors', it would be easier for humans to read and prepare the input data if whitespace was permitted on at least one side of the pipes. The difficulty of not accepting whitespace with pipes is also visible with the list of columns in the code example above.

Additional SQL Statements - Feature Request No. 2

I would like to be able to use the following SQL Statements such as:

  • GROUP BY <- most important for me
  • LIMIT
  • SELECT DISTINCT
  • SELECT COUNT
  • SELECT SUM

I do not think the Extension permits the use of these statements (or at least I do not know how to). Could these be added too please (or explained in Help Page)?

Thanks, Cruickshanks (talk) 10:34, 22 September 2014 (UTC)

Pleased to know that my extension is useful to somebody.
  • The bug is fixed in version 1.0.2 which I have just published.
  • Re Feature Request No. 1, I'm not totally sure what your point is. If you'd like a <datatbase2> tag to ignore whitespace around pipes, you can instruct it to consider it part of the field separator by adding the following attribute: fs="/\s*\|\s*/"
  • Re Feature Request No. 2, one would be tempted to make a fully-functional sql database out of this extension, since it would be logical to request a JOIN feature as soon as the GROUP BY feature has been implemented. I see a number of open questions there:
    1. How to express all this in attributes to <dt2-showtable> in a way that remains understandable.
    2. How to implement all this in code that remains maintainable.
    3. How to ensure efficiency of such queries.
I'm unlikely to find solutions in the near future. As an alternative, I'd suggest to use the Lua interface, query all records you need, and do further transformations on Lua level. This is clearly not efficient for large amounts of data, but the extension is not suitable for large amounts of data anayway. RV1971 (talk) 15:10, 24 September 2014 (UTC)
Thanks RV1971,
  • I have installed version 1.0.2 and fields and columns all correlate. Thanks for fixing.
  • Re Feature Request No. 1, I added the attribute: fs="/\s*\|\s*/" which did the trick. Earlier, I had simplistically tried things like " | " and "| " without success. Thanks for the help.
  • Re Feature Request No. 2, I understand the points you make. I discovered that the extension Extension:External_Data#.23get_db_data_-_retrieve_data_from_a_database works very well with the DataTable2 Extension and seems to provide the extra functionality I sought. Indeed, the other extension seems to have greater functionality than the author intended - see Extension_talk:External_Data#This_could_be_a_bug.
    • An additional trick I discovered by accident, is that Extension:DataTransclusion and Extension:External Data provide an easy way to look beyond/across the logical tables. This means that a large logical table can be created over several pages/templates provided that the same table structure is used with discipline. For example, using DataTable2 Extension separate templates could be set up for the members of football teams. Then using Extension:DataTransclusion and Extension:External Data, and the database column names like dtd_04, the entire list of players for the entire football league can be extracted.
    • This then means that the author has two choices: can either create and manage one table for the whole football league, or create and manage smaller tables for each football team. This could offer much utility for various Wikiprojects!
    • (I have used and like the impressive Semantic bundle too, but recognise that your extension does offer an alternative approach that suits my needs better.)
I regret not letting you know in my first post of another minor bug. It is a bug that is easily worked around. After first installing the extension and creating a page with the <datatable2> tag, the following error message was generated by both versions 1.0.1 and 1.0.2:
Warning: Cannot modify header information - headers already sent by (output started at /home/mysite/public_html/mw/extensions/DataTable2/DataTable2.php:277) in /home/mysite/public_html/mw/includes/WebResponse.php on line 37
As there are only 275 lines of code in DataTable2.php, deleting the two 'blank' lines (276 and 277) at the end of the file fixes the error. It seems that there is some form of invisible byte order mark on one or both of those lines that is causing the upset.
Regards, Cruickshanks (talk) 10:18, 26 September 2014 (UTC)
Thanks for spotting the bug re blank lines. Fixed in version 1.0.3. RV1971 (talk) 08:11, 29 September 2014 (UTC)

Define/query data in Semantic MediaWiki

... conceptually the converse of the Semantic MediaWiki extension and similar approaches. For instance, with Semantic MediaWiki you can store the population of a number of cities in the respective pages (hence one record per page) ...

I'd like to correct above statement in a sense that it is not precise in the context of using subobjects which allows to add many different facts on the same page. For example, you can store populations of a number of cities within the same page (if this is required) using the subobject datamodel and query those (as demonstrated in the economic data of a country table). MWJames (talk) 11:25, 22 September 2014 (UTC)

Thanks for the correction. I have modified the text accodingly. RV1971 (talk) 10:40, 24 September 2014 (UTC)

Using DataTAble2 in Templates

Hello again RV1971,

I am continuing to enjoy your extension and finding it a very powerful tool.

I am now trying to use it to gather up citations used throughout my wiki. I have tried various combinations of the following code added to the {{cite news |title= |url= |newspaper= |location= |publisher= |page= |date= |accessdate= }} template.

<datatable2 table="cite news" columns = "url|title|newspaper|location|date|accessdate|page|publisher" fs=";">
{{{url}}};{{{title}}};{{{newspaper}}};{{{location}}};{{{date}}};{{{accessdate}}};{{{page}}};{{{publisher}}}
</datatable2>

However, rather than insert the parameters into the datatable2-data table for each call to the {{Cite news}} template, it only injects {{{url}}}, etc. as text in the fields. This occurs when I save the actual {{Cite news}} template in the Template namespace, and nothing is generated by saving articles.

(As I could not get it to work, I do not know whether the table="cite news" parameter would simply overwrite the same row on each instance the template was called, sometimes multiple times in one article, and I would have to use one of the template parameters to create a unique logical table name.)

I realise that the DataTable2 extension uses the NewRevisionFromEditComplete hook to save data to the database, but with my low/fundamental knowledge was hoping that via substitution/transclusion this technique might be made to work when articles were saved with calls to the {{Cite news}} template.

Would you please provide me with guidance as to whether the proposed use of the DataTable2 extension might work with some further perserverence, or the concept is fundamentally flawed due to, for instance, the NewRevisionFromEditComplete hook. If the concept is flawed, I will drop my pursuit of this idea.

Thanks in advance. Cruickshanks (talk) 11:28, 14 October 2014 (UTC)

Dear Cruickshanks,
you're correct that the key point is the use of NewRevisionFromEditComplete. Indeed, it is the page text as saved in the database which is parsed for <datatable2> tags, without any kind of parameter expansion. It can't be otherwise because when saving the page, nothing is known about any transclusion within other pages.
So I'm afraid your idea needs to be dropped. RV1971 (talk) 19:39, 14 October 2014 (UTC)
Thanks for you quick response RV1971. Idea is dropped. :) Cruickshanks (talk) 00:12, 15 October 2014 (UTC)

Automatic page updates

Thanks for this great extension. Its going to become difficult to trust as the pages do not automatically update the pages the data is being pulled into when the table changes. I need to go to the target page, edit and click save. Is there something wrong with my configuration? 59.167.215.99 10:44, 2 March 2015 (UTC)

Pleased to know that my extension is useful. Whether your configuration is wrong depends on your needs: MediaWiki has various caching mechanisms, see Manual:Cache for an overview. Clearly you can reduce caching at the cost of performance loss. In any case, caching prevents automatic update of any underlying data; there is no way to see DataTable2 updates in real time but to allow other changes of underlying items (such as templates) to be deferred.
A simpler way to force update of a rendered page is explained in Manual:FAQ#How do I purge a cached page?. RV1971 (talk) 09:55, 3 March 2015 (UTC)

how to use dt2-expand the right way

Hello,

after upgrading to a recent mediawiki version I'm just migrating our product wiki from datatable to datatable2 and encountered a strange behaviour.

We show informations for one product on a single page, but store the data for all products on another page.

Our template for the single product page includes an includeonly part for e. g. assigning it to several product categories and a noinclude part for categorizing the template and including the Documentation template.

This is an abstracted version of the product page:

{{#dt2-expand:producttemplate|productable|pname = 'productname'}}

The product template contains

<includeonly>
[[category:snoozer|Vintage Pro 2000]]
</includeonly>
<noinclude>
{{documentation}}
[[category:template:layout|producttemplate]]
</noinclude>

With this enabled the product page will be categorized as template and the product template as normal page. When I swap the includeonly / noinclude tags everything is fine. Am I missing something in calling the dt2-expand function?

Thanks in advance,

Sabine ~o)sabine(o~ (talk) 17:20, 15 November 2015 (UTC)

Use standard MW table syntax?

Looks like an awesome extension...Was wondering though..Would it not be possible to allow the data tables to be expressed as normal Mediawiki tables, and enhanced as necessary through standard means? (e.g., through use of `data-*` attributes)

Another question--can the showtable tag accept or be made to accept normal template syntax?

I'd feel more comfortable putting my data into this format if it was only the showtable queries which required non-standard syntax (that, and for the sake of not needing as much work to migrate old tables). Brettz9 (talk) 02:57, 29 December 2015 (UTC)

Ajax tables

Just a feature request for Ajax-driven retrieval/filtering of sources if it might fit with your goals... Brettz9 (talk) 02:58, 29 December 2015 (UTC)

Update for MW 1.27.0

I just tried the extension in my MW 1.27.0 installation and when I tried to save my first table, I got an SQL exception. Some googling led me to https://phabricator.wikimedia.org/T120791 which suggests replacing the methods begin()/commit() with startAtomic()/endAtomic() respectively. I replaced them in the DataTable2Database.php file and the extension worked fine (I haven't tested all the features though). I would commit the update myself, but it's hosted at SourceForge where I'm not confortable. Could someone else take care? Sophivorus (talk) 15:27, 6 July 2016 (UTC)

Add Record with Extension:Page_Forms?

I use Extension:Page Forms to create new pages. Can a new entry be added to a DT2 table by PageForms extension? Johnywhy (talk) 09:38, 13 May 2018 (UTC)

I'm afraid I do not yet use Semantic MediaWiki, so I cannot tell for sure. I depends on whether Page Forms calls the NewRevisionFromEditComplete hook at the right moment. Just try it and let me know. If it does not work out of the box, perhaps I just need to add some other hook. RV1971 (talk) 17:23, 13 May 2018 (UTC)
i'm using Page Forms without Semantic MW. Johnywhy (talk) 07:26, 17 May 2018 (UTC)

Add Single Record From Other Page?

Possible to display textbox or input box on another page, to Add Single Record?

Extension:InputBox Johnywhy (talk) 09:24, 20 May 2018 (UTC)

I'm afraid your question is not clear to me. Could you kindly explain more in detail what you'd like to do? RV1971 (talk) 19:16, 27 May 2018 (UTC)
Display an input box on a page.
Type data into the inputbox.
Click Submit button, and a record gets added to a table, containing the data i typed.
Type new data into the inputbox.
Click Submit button, and another record gets added to the table. Johnywhy (talk) 00:28, 28 May 2018 (UTC)
DataTable2 works by parsing data from wikipage text. To make your idea work, you'd need an inputbox which inserts a record into the text of a page. I'm afraid I'm not aware of the existence of an inputbox of this kind. RV1971 (talk) 20:27, 3 June 2018 (UTC)

Extension does not work

Hello,

I'm running MediaWiki 1.31.1 on Mac OS and installed the "DataTable2" extension according to the instruction. For testing purposes, I added the code from the "demo table" example into a wikipage and received a fatal exception error of the typ "error".

Could you please be so kind and give advice? Thanks! Indolderivat (talk) 17:57, 13 November 2018 (UTC)

I'm afraid I'm still on MediaWiki 1.30.0. I hope to upgrade my installation in the next days. In the meantime, can you give any futher details about the error? RV1971 (talk) 19:19, 19 November 2018 (UTC)
I have just upgraded my MediaWiki to 1.31.1, and everything is working. So I'd need some further details to analyze this. You might take a look at Manual:How to debug. RV1971 (talk) 12:05, 21 November 2018 (UTC)
Thanks for your answer. I have reinstalled my wiki. Now I receive the following error code:
[08a362df8789b85d404df252] /index.php?title=Main_Page&action=submit Error from line 455 of .../Wiki/extensions/DataTable2/includes/DataTable2.php: Call to undefined function wfProfileIn()
Backtrace:
#0 .../Wiki/includes/parser/Parser.php(3888): DataTable2->renderDataTable(string, array, Parser, PPFrame_DOM)
#1 .../Wiki/includes/parser/Preprocessor_DOM.php(1362): Parser->extensionSubstitution(array, PPFrame_DOM)
#2 .../Wiki/includes/parser/Parser.php(2942): PPFrame_DOM->expand(DOMElement, integer)
#3 .../Wiki/includes/parser/Parser.php(1293): Parser->replaceVariables(string)
#4 .../Wiki/includes/parser/Parser.php(443): Parser->internalParse(string)
#5 .../Wiki/includes/content/WikitextContent.php(323): Parser->parse(string, Title, ParserOptions, boolean, boolean, NULL)
#6 .../Wiki/includes/content/AbstractContent.php(516): WikitextContent->fillParserOutput(Title, NULL, ParserOptions, boolean, ParserOutput)
#7 .../Wiki/includes/page/WikiPage.php(2129): AbstractContent->getParserOutput(Title, NULL, ParserOptions)
#8 .../Wiki/includes/page/WikiPage.php(1655): WikiPage->prepareContentForEdit(WikitextContent, NULL, User, string, boolean)
#9 .../Wiki/includes/EditPage.php(2214): WikiPage->doEditContent(WikitextContent, string, integer, boolean, User, string, array, integer)
#10 .../Wiki/includes/EditPage.php(1506): EditPage->internalAttemptSave(array, boolean)
#11 .../Wiki/includes/EditPage.php(652): EditPage->attemptSave(array)
#12 .../Wiki/includes/actions/EditAction.php(60): EditPage->edit()
#13 .../Wiki/includes/actions/SubmitAction.php(38): EditAction->show()
#14 .../Wiki/includes/MediaWiki.php(500): SubmitAction->show()
#15 .../Wiki/includes/MediaWiki.php(294): MediaWiki->performAction(Article, Title)
#16 .../Wiki/includes/MediaWiki.php(861): MediaWiki->performRequest()
#17 .../Wiki/includes/MediaWiki.php(524): MediaWiki->main()
#18 .../Wiki/index.php(42): MediaWiki->run()
#19 {main}
Indolderivat (talk) 14:43, 21 November 2018 (UTC)
You are using an old version of the extension, the calls to wfProfileIn() have been removed in May 2018. Pls try downloading the current version. RV1971 (talk) 15:18, 21 November 2018 (UTC)
I downloaded the following version "DataTable2-REL1_31-c178d87.tar" via thisLink. My assumption was that I will receive an acutal version. Could you please advice how to proceed? Thanks again for your support! Indolderivat (talk) 18:35, 26 November 2018 (UTC)
I'm afraid you're perfectly right! Sorry, I failed to merge the master branch into the REL1_31 branch. I have just done that, so a corrected version should be available soon at that link. RV1971 (talk) 23:22, 26 November 2018 (UTC)
Thanks for uploading the new version. I installed it today and checked its presence under Special:Version. The extension was listed there. However, when I tested the <datatable2>-example code from extension page, the following error was generated: Fatal exception of type "Wikimedia\Rdbms\DBQueryError". Any thought on it? Indolderivat (talk) 18:14, 29 November 2018 (UTC)
I just realized that in some older edit on #Installation I removed by mistake the information that a database upgrade is needed to install the extension, so I guess you didn't do this. I have just corrected the instructions. Pls run the database upgrade script. RV1971 (talk) 11:28, 1 December 2018 (UTC)
Perfect. It works now. Thanks a lot! Indolderivat (talk) 21:01, 5 December 2018 (UTC)

Including table head in external template

I have been playing with this extension trying to use it with the template in an external page, and then passing the template name as an argument. My problem is that I would also like to have the header included in the template. Is there a way to include the head definition within the template as well so I can externalize both the head and rows from the call. I would like to do something like:

<dt2-showtable table="Table1" template="Template:Table1 Template" class="wikitable" args="Descriptor=Population" where="State in ('Wisconsin')">


I was hoping that the external template would work similar to the inline template where you can pass a <head> tag so I could do something similar to below as my template but it doesn't seem to be working:

<head>

|-

! rowspan="2" | State || colspan="3" |{{{Descriptor}}} Counts || colspan="3" | Change in # of {{{Descriptor}}}||colspan="3" | % Change in {{{Descriptor}}}

|-

! 2017 || 2018 || 2019 || 2017 to 2018 || 2018 to 2019 || 2017 to 2019 || 2017 to 2018 || 2018 to 2019 || 2017 to 2019

</head>

|-

! {{{Business_Category}}} | {{{CY2017}}} || {{{CY2018}}} || {{{CY2019}}}

| {{{2017_to_2018_delta}}} || {{{2018_to_2019_delta}}} || {{{2017_to_2019_delta}}}

| {{{Percent_Change(2017-2018)}}} || {{{Percent_Change(2018-2019)}}} || {{{Percent_Change(2017-2019)}}}

Am I missing something, or is it not possible to define a <head> within an external template?

Thanks 161.185.208.127 (talk) 14:47, 25 August 2020 (UTC)

I'm not sure if I understand what you mean. If I get it right, you'd like to include a template that contains <head>...</head>. This cannot work because the content in <dt2-showtable>...</dt2-showtable> is first parsed for things like <head>...</head>, and then any template substitution takes place. Hope this answers your question. RV1971 (talk) 20:56, 28 August 2020 (UTC)

Errors when defining a table


Deprecated: Use of Revision::__construct was deprecated in MediaWiki 1.31. [Called from MediaWiki\Storage\PageUpdater::doModify in /var/www/html/includes/Storage/PageUpdater.php at line 1252] in /var/www/html/includes/debug/MWDebug.php on line 376

Deprecated: Use of NewRevisionFromEditComplete hook (used in onNewRevisionFromEditComplete) was deprecated in MediaWiki 1.35. [Called from MediaWiki\HookContainer\HookContainer::run in /var/www/html/includes/HookContainer/HookContainer.php at line 137] in /var/www/html/includes/debug/MWDebug.php on line 376

Deprecated: Use of Revision::getContent was deprecated in MediaWiki 1.31. [Called from DataTable2::onNewRevisionFromEditComplete in /var/www/html/extensions/DataTable2/includes/DataTable2.php at line 260] in /var/www/html/includes/debug/MWDebug.php on line 376

I downloaded the version after selecting the 1.36 MW version. 84.111.224.227 (talk) 13:00, 11 January 2022 (UTC)

I have just updated my MediaWiki to 1.37 and pushed a fix for DataTable2 so that it now works with 1.37. So you might try upgrading to 1.37. RV1971 (talk) 14:11, 15 January 2022 (UTC)

mw.ext.datatable2 is nil

I'm loading DataTable2 after Scribunto, both extensions show up in the versions page, mw.ext.ParserFunctions does exist. 84.111.224.227 (talk) 14:31, 11 January 2022 (UTC)

I think this has to do with the check 
if ( isset( $wgExtensionCredits['parserhook']['Scribunto'] ) ) {
wgExtensionCredits doesn't appear in the Scribunto code 84.111.224.227 (talk) 15:09, 11 January 2022 (UTC)

Maximum length

I tried to put a very long paragraph of sentences in a data table and it appears to be truncated at 255 characters. Is this a fixed limit? Is there any way to have longer lengths? 70.44.167.109 (talk) 16:21, 14 March 2022 (UTC)

@RV1971 just wanted to ping you so you see this. Thanks for your hard work on this extension! Please see my question above when you have a moment to reply. Thanks! 70.44.167.109 (talk) 16:22, 14 March 2022 (UTC)
Indeed this limit is hard-wired in the table definition, see datatable2_data.sql. You can change it by modifying the SQL file before installing the extension. If you have already installed it, you can still modify the table in your database. In case you don't know how to do this, look for ALTER TABLE in the documentation of your underlying database system. RV1971 (talk) 22:19, 15 March 2022 (UTC)
@RV1971
That was what I thought. I was about to change the table definition using my hosting provider's phpmyadmin web interface but was concerned that it may lead to other issues. I looked over your code via github and I can't see any reason why it wouldn't work, but just wanted to pick your brain on it before I tried it and broke something. I'm gonna up it to 1023 I think. Thanks for the help! 70.44.167.109 (talk) 23:54, 16 March 2022 (UTC)
That limit might be worth mentioning in the extension documentation. 70.44.167.109 (talk) 23:54, 16 March 2022 (UTC)
Good idea. I have just done so, thanks! RV1971 (talk) 20:44, 18 March 2022 (UTC)

dt2-showtable using in template page

When use dt2-showtable tag in template page, the variables in template could not convert to value.

In the old discussion, I found out that datatable2 tag cannot be used to store data in template. I think it's right.However, it may be work for dt2-showtable tag use in template. Such as add:

$args['where']=$parser->replaceVariables($args['where'], $frame, true )

in '''renderShowTable''' to convert '''where''', '''order-by'''. Is this suitable? Sunwui (talk) 15:11, 6 September 2023 (UTC)

You can use the #tag magic word to achieve this. RV1971 (talk) 20:50, 6 September 2023 (UTC)
Thanks! I try to use #tag magic word for dt2-showtable in template, it work!
I try to use for datatable2 also, even if not set variable, such as: {{#tag:datatable2|{{FULLPAGENAME}};interesting page;extension description|table=demo1|class=wikitable|columns=page{{!}}rating{{!}}type|fs=;}}, it can show but no data store to data table, so it only work for show table? Sunwui (talk) 03:35, 7 September 2023 (UTC)
Yes, by the way MW works, data can be stored with the tag itself only. RV1971 (talk) 07:20, 7 September 2023 (UTC)

Looking for a reviewer to make it work in MW 1.42

The special pages in this extension do not currently work in MW 1.42. A trivial change that fixes this is available in DataTable2TablesPager: Fix getQueryInfo() to use Wikimedia\Rdbms\Subquery. Could someone kindly review this change? RV1971 (talk) 15:30, 2 December 2024 (UTC)