What is the reason for the arbitrary changes from SQL to this proposed language, for example, the addition of braces? I could imagine a rationale for using a completely different language, like XML, but I don't really see the rationale for taking SQL and changing the grammar in arbitrary ways that make it more difficult to port the existing patch files.
Topic on Talk:Requests for comment/Abstract table definitions
The aim was to have a well defined, simple, and readable DSL for this. The braces are just a flavor you could bike-shed over but the general idea is to be able to simply declare say a table with individual lines nested inside of it defining the columns. It's not SQL, it's just declaring things.
I didn't use SQL because that would basically involve re-implementing parsing of SQL in PHP just to map to Database class methods. It would be unnecessarily complex to implement and wouldn't properly cover the important subtleties to our database structure. And it wouldn't be very readable or cover things like the fact that practically every column is NOT NULL making declaring that excessive and unfriendly to people defining new columns and tables.
While declaration can be done with something like XML or PHP arrays that kind of thing has already been attempted. And I found that these just end up extremely verbose. They are extremely hard to read and understand what's going on, even worse than our SQL files.
As for porting existing things. That will never be easy. These table definitions and modifiers aren't completely equivalent to any single sql patch or file we have. Our tables have differences between database engines that go beyond simple SQL syntax differences and simple type differences. We use references/foreign keys in some engines but not in others. And it's impossible to declare a mapping using types from our declarations for any one database engine that will consistently map to the types we use in other database engines. So porting migrations and tables will always require thought and effort no matter what the syntax.
It's defensible, I suppose. I'd like to hear other opinions on the details of the language from people who have dealt with this area of the code, such as Chad Horohoe, OverlordQ and Max Semenik.
I indeed attempted this with a PHP array before, together with Chad. We ended up abandoning this effort to avoid delaying the new installer infinitely.
My main concern is that this change should make things easier, not harder. The more elaborate the DSL is the greater is the chance that we will simply move the complexity to other part of the software in the longer run - DB updates might become easier but now we'll have to maintain a huge DSL abstraction layer. Before supporting this RFC, I would like an assesment of how cumbersome will the implementation be.