Topic on Project:Support desk

update.php Error: 1054 Unknown column 'ar_comment_id' in 'where clause' (localhost)

33
Beardedfool (talkcontribs)

Seems to be a new field, should this be covered in update.php?


Trying to migrate to a new install of 1.33.0 on a new server from 1.32.1.

Mysql dump


On running update.php I'm getting the above. Manually running migrateactors.php doesn't seem to do it either.


The way our setup works it's a bit awkward to do it another way but guess I can install 1.32.1 on the new server and patch up if needed. But should I have to please?


It could also be something weird in our LocalSettings.php as this has built up over time and I'm aiming to start again, but wanted to rule out the update.php and ask for advice first please.


TheDJ (talkcontribs)

This might be a bug.. I filed https://phabricator.wikimedia.org/T227662 to which you can subscribe.

Do you have any migrate settings specified in your LocalSettings.php ? And is this mysql/mariadb, or something else ?

81.97.98.8 (talkcontribs)

Thanks for filling that.

10.1.38-MariaDB-0ubuntu0.18.10.2 Ubuntu 18.10

10.1.40-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04

Can you talk me through the migrate settings part? At the extent of my knowledge here

Iowajason (talkcontribs)

Similar problem with 1.32.x to 1.33 upgrade of migrateComments script, except on column 'pt_reason_id'. Checked MySQL schema and column does not exist in database. DB is MySQL Community 8.0.16. No settings in LocalSettings that seem to be migration related.

TheDJ (talkcontribs)

So this is reporting errors on database columns that were introduced in 1.30. Have both of you ran pre-1.30 versions at some point in time ?

Iowajason (talkcontribs)

My Mediawiki install definitely predates 1.30. Clearly recall 1.22 to 1.23 upgrade challenges and think might have been 1.08 or so for initial install.

TheDJ (talkcontribs)

Also, you seem to have a comment table, can you check what the structure of that table is and report back here (or in the ticket).

Iowajason (talkcontribs)

MySQL DDL reports:

CREATE TABLE `comment` (

  `comment_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,

  `comment_hash` int(11) NOT NULL,

  `comment_text` blob NOT NULL,

  `comment_data` blob,

  PRIMARY KEY (`comment_id`),

  KEY `comment_hash` (`comment_hash`)

) ENGINE=InnoDB AUTO_INCREMENT=8651 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci

Iowajason (talkcontribs)

My pre-upgrade (May) backup version also doesn't seem to have for pt_reason_id flavor of issue:

CREATE TABLE `protected_titles` (

  `pt_namespace` int(11) NOT NULL,

  `pt_title` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

  `pt_user` int(10) unsigned NOT NULL,

  `pt_reason` tinyblob,

  `pt_timestamp` binary(14) NOT NULL,

  `pt_expiry` varbinary(14) NOT NULL DEFAULT '',

  `pt_create_perm` varbinary(60) NOT NULL,

  PRIMARY KEY (`pt_namespace`,`pt_title`),

  KEY `pt_timestamp` (`pt_timestamp`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;

Bttfvgo (talkcontribs)

I have the exact same problem! I started out using 1.29 (followed by 1.30) and just updated to 1.32. Everything seemed to be working and update.php worked great. I then tried updating to 1.33 and ran into the same problem... "Error: 1054 Unknown column 'ar_comment_id' in 'where clause' (localhost)". I'm dumbfounded. I did have the Comments extension at one time but haven't used it in years. Not sure how to get table structure. Is this what you mean? Do I need to resort back to 1.32? Is it because the Comments extension hasn't been updated in years because I disabled it? I am SO glad it isn't just me having this problem!

mysql> DESCRIBE (or EXPLAIN) comment;
+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| comment_id   | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| comment_hash | int (11)            | NO   | MUL | NULL    |                |
| comment_text | blob                | NO   |     | NULL    |                |
| comment_data | blob                | YES  |     | NULL    |                |
+--------------+---------------------+------+-----+---------+----------------+
Bttfvgo (talkcontribs)

I also have a table "Comments", rather than "comment", but I'm sure the error refers to the latter.

When I use "describe archive;" I get all of the "ar_" fields (ar_comment is one of them) but the only names ending in "_id" are "ar_id", "ar_rev_id", "ar_text_id", "ar_page_id", and "ar_parent_id". No "ar_comment_id".

Bttfvgo (talkcontribs)

Looking at the page Manual:Archive_table, my archives table looks identical to the MediaWiki versions 1.25 – 1.29 table. So I don't have the column they are wanting to migrate!  :( Can I create the fields myself?

TheDJ (talkcontribs)

Anyone of you have a backup to check if you had those _id columns before upgrading ?

The weird thing is that those *_id's should have been added at the same time as the comment table was added (and that comment table describe of Bbttfvgo does look correct). Also weird is that some of you have different missing fields than the others..

So either something went wrong in originally adding them during 1.30 or they later got dropped, or there is an error in the migration script or the 1.33 sql updates..

Iowajason (talkcontribs)

My Backup from May of this year has (seemingly identical) schema of:

CREATE TABLE `comment` (

  `comment_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,

  `comment_hash` int(11) NOT NULL,

  `comment_text` blob NOT NULL,

  `comment_data` blob,

  PRIMARY KEY (`comment_id`),

  KEY `comment_hash` (`comment_hash`)

) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;

/*!40101 SET character_set_client = @saved_cs_client */;

Iowajason (talkcontribs)

I didn't experience problem with that column or several that other reports listed. I did have ar_comment_id, ipb_reason_id, fa_deleted_reason_id, and log_comment_id in my May backup. My Wiki just missing the protected titles ID column. Didn't lose other columns due to 1.33 update, just page display seems to now depend on some of those columns.

Bttfvgo (talkcontribs)

Okay I went through my DB backup from MW 1.30 (dated 8-17-2018) and the archives table is identical to the one I have now. Well with one exception. The field named "ar_text" was in the 1.30 backup but now no longer exists.

Bttfvgo (talkcontribs)

Can we just recreate the missing table fields? Should I switch back to an older version and try the migrateComments.php command?

I went back through a DB backup from MW 1.29 (dated 5-17-2017) and the archive table is there (with the same 19 fields) but the comment table is not.

Bttfvgo (talkcontribs)

Okay, so I decided to:

mysql> ALTER TABLE archive
    -> ADD COLUMN ar_comment_id INT NOT NULL;

and ran the upgrade script again. This time it actually gave me a different error:

Error: 1054 Unknown column 'ipb_reason_id' in 'where clause' (localhost)

So if I can create all of the columns it claims it is missing, do you think it might actually go through? I'll test the theory, missing file by missing file, and let you know what happens!

Bttfvgo (talkcontribs)

Okay so I did:

mysql> ALTER TABLE ipblocks
    -> ADD COLUMN ipb_reason_id BIGINT NOT NULL;

and ran the upgrade script and it passed this stage (updated 1039 row(s) with 7 new comment(s)). The next step, 'img_description_id' went through with no problem. (updated 2893 row(s) with 6 new comment(s)). After this I get the error:

Error: 1054 Unknown column 'oi_description_id' in 'where clause' (localhost)

. I'll add this and let you know how it goes!

Bttfvgo (talkcontribs)
mysql> ALTER TABLE oldimage
    -> ADD COLUMN oi_description_id BIGINT NOT NULL;

and that worked too (updated 121 row(s) with 6 new comment(s)). Next error is 'fa_deleted_reason_id'. Will report back soon.

Bttfvgo (talkcontribs)

fa_deleted_reason_id

mysql> ALTER TABLE filearchive
    -> ADD COLUMN fa_deleted_reason_id BIGINT NOT NULL;

led to fa_description_id

mysql> ALTER TABLE filearchive
    -> ADD COLUMN fa_description_id BIGINT NOT NULL;

which led to rc_comment_id

mysql> ALTER TABLE recentchanges
    -> ADD COLUMN rc_comment_id BIGINT NOT NULL;

which led to log_comment_id

mysql> ALTER TABLE logging
    -> ADD COLUMN log_comment_id BIGINT NOT NULL;

which led to pt_reason_id

mysql> ALTER TABLE protected_titles
    -> ADD COLUMN pt_reason_id BIGINT NOT NULL;

My wiki's pretty big so the last two took forever to update but IT WORKS NOW! YEA!!! Hopefully these findings will help many more users, including the two whose thread I inadvertantly hijacked. Thanks again for a superb and superior product. Mediawiki for life!

Rainer Prem (talkcontribs)

Thank you very much for this post. But in my case (updating from 1.18) there were even more columns missing. So the total script is this:


ALTER TABLE archive ADD COLUMN ar_comment_id INT NOT NULL; ALTER TABLE ipblocks ADD COLUMN ipb_reason_id BIGINT NOT NULL; ALTER TABLE oldimage ADD COLUMN oi_description_id BIGINT NOT NULL; ALTER TABLE filearchive ADD COLUMN fa_deleted_reason_id BIGINT NOT NULL; ALTER TABLE filearchive ADD COLUMN fa_description_id BIGINT NOT NULL; ALTER TABLE recentchanges ADD COLUMN rc_comment_id BIGINT NOT NULL; ALTER TABLE logging ADD COLUMN log_comment_id BIGINT NOT NULL; ALTER TABLE protected_titles ADD COLUMN pt_reason_id BIGINT NOT NULL; ALTER TABLE archive ADD COLUMN ar_actor INT NOT NULL; ALTER TABLE ipblocks ADD COLUMN ipb_by_actor int NOT NULL; ALTER TABLE image ADD COLUMN img_actor int NOT NULL; ALTER TABLE oldimage ADD COLUMN oi_actor INT NOT NULL; ALTER TABLE filearchive ADD COLUMN fa_actor INT NOT NULL; ALTER TABLE recentchanges ADD COLUMN rc_actor INT NOT NULL; ALTER TABLE logging ADD COLUMN log_actor INT NOT NULL;

186.210.41.56 (talkcontribs)

Really nice, thank you so much this worked like a charm.

Rainer Prem (talkcontribs)

Oops. Can I change the rpevious post?

Beardedfool (talkcontribs)

Zero problem on the hijack as you seem to have an answer, thanks! Though I'll wait to for the more official answer of what to do.


@TheDJ The version before the 1.32.1 was so old it didn't have the comment part in it. Not sure of what version that was I'm afraid


10.1.38-MariaDB-0ubuntu0.18.10.2 Ubuntu 18.10 (version 1.32.1)

+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| comment_id   | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| comment_hash | int(11)             | NO   | MUL | NULL    |                |
| comment_text | blob                | NO   |     | NULL    |                |
| comment_data | blob                | YES  |     | NULL    |                |
+--------------+---------------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)


10.1.40-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04 (version 1.33.0)

MariaDB [wiki]> DESCRIBE wiki_comment;
+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| comment_id   | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| comment_hash | int(11)             | NO   | MUL | NULL    |                |
| comment_text | blob                | NO   |     | NULL    |                |
| comment_data | blob                | YES  |     | NULL    |                |
+--------------+---------------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
TheDJ (talkcontribs)

Right, so it seems all of you had the comment table since 1.30, but not the new *_id columns in the other tables. That likely means no accidental data loss or something scary like that, but it is still very weird, as the table and the new _id fields of the other columns were added in the same SQL change file..

None of you remember running into an error with previous upgrades ? Did any of you run git versions of MediaWiki perhaps ?

The changes that SHOULD have run, but apparently for some of you didn't run completely are: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/maintenance/archives/patch-comment-table.sql

If you make backups etc and manually try to run these instructions, then maybe you can figure out where things went wrong.. Please don't do things you are not comfortable with. P.S. The /**/ syntax the SQL commands is where the table prefix generally is inserted (often that's just wiki_)

Beardedfool (talkcontribs)

Didn't see any errors before - wouldn't want to bet my life on that but pretty sure. When I get a chance I'll try to setup a new one to test that for you.

I'm not totally comfortable with bits in SQL side but will try those as I have backups and am happy enough dropping the database entirely, Mariadb in necessary. The server isn't doing anything else at present.


Unlikely to be today though, hopefully tomorrow. Thanks for the help!


Couple of questions for my learning please?

So that's mostly copy and enter in SQL command line after sed /*_*/ with wiki_ e.g /*_*/comment becomes wiki_comment

1) What's /*i*/ on lines 12/19?

2) On the CREATE INDEX lines, what does the last part of the line in brackets do e.g. (imgcomment_name) - column name?

Jagatronic (talkcontribs)

I think this may be collateral damage from the "max key length is 767 bytes" issue with MySQL / UTF support / varchar(255) fields. I hit the max key length problem upgrading to 1.31.

Now trying to upgrade 1.32.1 -> 1.33 I hit this bug. Trying the SQL from patch-comments-table.sql line by line it fails at "CREATE TABLE /*_*/image_comment_temp" because of the max key length problem. It looks like none of the rest of this file was applied.

Are there likely to be other bits of updates that have been missed along the way? Is there some way to validate the whole schema is up to date?

Jagatronic (talkcontribs)

Having fixed my max length problem and applied everything in patch-comments-table.sql the update progressed until I hit Topic:V3adkmqnefp8bf30 - this is where I first hit the max key length bug a couple of upgrades ago. It turns out my database doesn't have the changes in maintenance/archive/patch-actor-table.sql either (except for the first table: actor).


Iowajason (talkcontribs)

Reran "patch-comment-table.sql" using MySQL Workbench and received SQL syntax error (recall I got further than others with 1.31 update and protected titles the only issue I'm trying to resolve).


SQL:

INSERT OR IGNORE INTO /*_*/protected_titles_tmp (  pt_namespace, pt_title, pt_user, pt_reason, pt_timestamp, pt_expiry, pt_create_perm)   SELECT  pt_namespace, pt_title, pt_user, pt_reason, pt_timestamp, pt_expiry, pt_create_perm   FROM /*_*/protected_titles

ErrirL

Error Code: 1064. You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OR IGNORE INTO /*_*/protected_titles_tmp (  pt_namespace, pt_title, pt_user, pt_' at line 1 0.079 sec


MySQL didn't like the "OR" command, said "expecting identifier."


This SQL worked (note zero rows since protection not used on my site):

INSERT INTO protected_titles_tmp (

pt_namespace, pt_title, pt_user, pt_reason, pt_timestamp, pt_expiry, pt_create_perm)

  SELECT

pt_namespace, pt_title, pt_user, pt_reason, pt_timestamp, pt_expiry, pt_create_perm

  FROM protected_titles;

Ran rest of file:

DROP TABLE /*_*/protected_titles;

ALTER TABLE /*_*/protected_titles_tmp RENAME TO /*_*/protected_titles;

CREATE UNIQUE INDEX /*i*/pt_namespace_title ON /*_*/protected_titles (pt_namespace,pt_title);

CREATE INDEX /*i*/pt_timestamp ON /*_*/protected_titles (pt_timestamp);

COMMIT;


Then encountered the actor table migration problem mentioned previously.


TheDJ (talkcontribs)

@Iowajason You ran the sqlite version instead of the mysql version

Beardedfool (talkcontribs)

Has everyone else got this sorted now. Tried running the patch-comment-table.sql and all seemed to work OK but still get the below


I can't work out from the phabricator link whether this is being dealt with in an update or my next steps


Migrating actors to the 'actor' table, printing progress markers. For large

databases, you may want to hit Ctrl-C and do this manually with

maintenance/migrateActors.php.

Creating actor entries for all registered users

... 1 - 101

... 102 - 202

... 203 - 236

Completed actor creation, added 0 new actor(s)

Beginning migration of revision.rev_user and revision.rev_user_text to revision_actor_temp.revactor_actor

Completed migration, updated 0 row(s) with 0 new actor(s), 0 error(s)

Beginning migration of archive.ar_user and archive.ar_user_text to archive.ar_actor

An error occurred:

A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?

Query: SELECT  ar_id,ar_user,ar_user_text,CASE WHEN ar_user = 0 OR ar_user IS NULL THEN (SELECT  actor_id  FROM `wiki_actor`    WHERE (ar_user_text = actor_name)  ) ELSE (SELECT  actor_id  FROM `wiki_actor`    WHERE (ar_user = actor_user)  ) END AS `actor_id`  FROM `wiki_archive`    WHERE ar_actor = '0' AND (1=1)  ORDER BY ar_id LIMIT 100  

Function: MigrateActors::migrate

Error: 1054 Unknown column 'ar_actor' in 'where clause' (localhost)

Reply to "update.php Error: 1054 Unknown column 'ar_comment_id' in 'where clause' (localhost)"