Meza/Directory structure

This is an overview of the directory structure used by a meza server. This does not include every single file, but attempts to list out directories and files used by the meza application.

Info required: which files exist on which server types? Currently this is mostly focused on monolithic installations, where the controller and all applications (MariaDB, Elasticsearch, MediaWiki, etc) are on the same server.

Glossary
There are several terms in this document which may be described in the [Glossary](Glossary.md).

Major directories
The five major directories are:


 * : The meza application. Only modified when developing for or upgrading meza version
 * : meza configuration. Modified by sysadmins when making configuration changes.
 * : storage of meza information. Modified anytime wiki users do anything on the site.
 * : Files deployed by the meza application. Sysadmins don't need to touch this. Instead, modify  or   and re-deploy.
 * : The webserver root. Deployed just like.

Meza application directory. This should never change except if you upgrade meza (or you're doing development for the meza application itself). This only exists on the controller.


 * : Ansible config file. Ref: http://docs.ansible.com/ansible/intro_configuration.html
 * : base config vars, able to be overridden in /opt/conf-meza/public/public.yml
 * : mostly unused attempts to internationalize meza
 * : extensions installed on all meza servers
 * : CentOS7 VirtualBox Host-Only network interface template
 * : Ref Issue #712: Flatten config/core/ to just config/
 * : Directions on how to contribute to meza
 * : Config file for CodeClimate analysis
 * : Config file for CodeClimate analysis
 * : Template text for new GitHub issues
 * : Template text for new GitHub pull requests
 * : Ref: https://git-scm.com/docs/gitignore
 * : license info for meza
 * : GIF for main README showing usage of meza command
 * : ASCII art of pigeon declaring successful install; Ref Issue #676: reincorporate into deploy
 * (files used for help when using meza command, e.g. )
 * (various files explaining various aspects of meza)
 * : meza's main README
 * (Ansible playbooks. Ref: http://docs.ansible.com/ansible/playbooks.html)
 * (Ansible roles. Ref: http://docs.ansible.com/ansible/playbooks_roles.html)
 * Run on hosts: : Create a Vbox VM; remove when   mature? Or keep for RedHat
 * Setup dev VMs: : Setup host-only network on Vbox VM; Remove after Vagrant? Keep for RedHat setup?
 * Run on controller: : (Used to install   command)
 * Run on controller: : (Entry point for   command)
 * Consolidate :  :
 * Consolidate :  :
 * : Ref Issue #714: Move to tests/integration/ and use in automated testing
 * : Ref Issue #672: determine if still functional
 * : Scripts run on controllers which run several commands to test various functions
 * : Ansible playbook run on controller to setup a test case. creates a fake non-meza source to import from.
 * : Scripts run on a Docker host
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates a generic container
 * : uses init-container to generate a controller container
 * : uses init-container to generate a minion container
 * : Entrypoint script to setup which tests to run. first argument for this script is which test type to run.
 * : finds image URL via JS API, the checks is present on server
 * : several basic tests to verify a meza installation is functioning
 * : checks wiki API, Parsoid, Elasticsearch for a wiki
 * : shim to handle git checkout funnies in Travis
 * : Config file for Travis CI automated testing
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/
 * : Scripts run on controllers which run several commands to test various functions
 * : Ansible playbook run on controller to setup a test case. creates a fake non-meza source to import from.
 * : Scripts run on a Docker host
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates a generic container
 * : uses init-container to generate a controller container
 * : uses init-container to generate a minion container
 * : Entrypoint script to setup which tests to run. first argument for this script is which test type to run.
 * : finds image URL via JS API, the checks is present on server
 * : several basic tests to verify a meza installation is functioning
 * : checks wiki API, Parsoid, Elasticsearch for a wiki
 * : shim to handle git checkout funnies in Travis
 * : Config file for Travis CI automated testing
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/
 * : several basic tests to verify a meza installation is functioning
 * : checks wiki API, Parsoid, Elasticsearch for a wiki
 * : shim to handle git checkout funnies in Travis
 * : Config file for Travis CI automated testing
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/

These are shell scripts used to create users. Ref Issue #713: Consolidate



Location of configuration setup for meza. This is split into  and   config. Secret config is intended for sensitive info like passwords. It also houses your  file which declares which servers have which components installed (e.g. Parsoid is installed on  ). The  file may or may not be considered sensitive, but it resides in   nonetheless. Some non-sensitive items may be more convenient to put in. These include things like, which you may only want set to   on your production setup and no others. Since the  file resides in   those environment-specific settings may be better to keep in.

Additionally, the user  is used by meza to perform most actions. Some other actions are performed by the  user. However, we don't create these users in  due to possible conflicts with other user systems. Ref #727.


 * : Use to define extra extensions for your meza installation
 * : MAIN CONFIGURATION VARIABLES FILE!
 * (Any .php files here will be loaded at the end of LocalSettings.php)
 * (Any .php files here will be loaded at the beginning of LocalSettings.php)
 * : this "demo" is known as the wiki ID
 * : favicon
 * : logo
 * ( files here loaded at end of LocalSettings.php for this wiki only)
 * ( files here loaded at beginning of LocalSettings.php for this wiki only)
 * (more wikis with same format as above)
 * : This is the main secret config file. Encrypted by vault password.
 * : AKA "Inventory file", listing server roles. See [Glossary](Glossary.md) for more info
 * : SSL certificate for this environment. Encrypted by vault password
 * : Private key for this environment. Encrypted by vault password
 * (more enviroments if you so choose)
 * : secret key file on controller only. KEEP SAFE!
 * : public key file. This needs to be put on any minion servers.
 * : vault password file for monolith environment. See /opt/conf-meza above. On controller only
 * : (vault password file for other environments)
 * : alternate user for some operations to avoid conflicts
 * : AKA "Inventory file", listing server roles. See [Glossary](Glossary.md) for more info
 * : SSL certificate for this environment. Encrypted by vault password
 * : Private key for this environment. Encrypted by vault password
 * (more enviroments if you so choose)
 * : secret key file on controller only. KEEP SAFE!
 * : public key file. This needs to be put on any minion servers.
 * : vault password file for monolith environment. See /opt/conf-meza above. On controller only
 * : (vault password file for other environments)
 * : alternate user for some operations to avoid conflicts
 * : public key file. This needs to be put on any minion servers.
 * : vault password file for monolith environment. See /opt/conf-meza above. On controller only
 * : (vault password file for other environments)
 * : alternate user for some operations to avoid conflicts

=  =

Data storage for meza. This is basically any information generated by using meza, including MariaDB (MySQL) database info, files uploaded by users, logs from meza commands (logs from Apache, HAProxy, system logs, etc are kept in their standard CentOS/RHEL locations), temp files, and backups from running  command.


 * : backups directory populated from running
 * : backups for the "monolith" environment
 * : database backup from specified timestamp
 * : backup of uploads directory
 * : backups for the "production" environment
 * (wikis in production)
 * : files associated with Elasticsearch data
 * : directory holding error logs related to PHP
 * : directory for logs from automated job runner
 * : To be used by Meza/Autodeployer
 * : To be used by Autodeployer
 * : TBD
 * : Logs for backup cleanups, uploadstash cleanup, etc
 * : log from nightly parsoid restarts
 * : log from this wiki's search index rebuilding
 * : log from this wiki's SMW rebuildData.php
 * : many files and directories associated with MariaDB data
 * : Ref Issue #613: mysql-bin files are not cleaned up by meza currently
 * : temp files which can be deleted. Ref Issue #497: need to keep this directory clean
 * : location where Demo Wiki's uploads/images are kept
 * (more wikis uploads/ directories)
 * : used instead of  if multiple app-servers used. File system distributed between app-servers.
 * : temp files which can be deleted. Ref Issue #497: need to keep this directory clean
 * : location where Demo Wiki's uploads/images are kept
 * (more wikis uploads/ directories)
 * : used instead of  if multiple app-servers used. File system distributed between app-servers.
 * (more wikis uploads/ directories)
 * : used instead of  if multiple app-servers used. File system distributed between app-servers.

This directory is a hidden directory (e.g. starts with a period) because it really shouldn't be modified directly. When the  command is run this directory is set based upon the config set in.


 * : PHP config variable file written based off defaults, secret and public config
 * : Bash config variable file written based off defaults, secret and public config
 * : Extensions to load, written based off core and local extensions
 * : ReThis is an overview of the directory structure used by a meza server. This does not include every single file, but attempts to list out directories and files used by the meza application.

Info required: which files exist on which server types? Currently this is mostly focused on monolithic installations, where the controller and all applications (MariaDB, Elasticsearch, MediaWiki, etc) are on the same server.

Glossary
There are several terms in this document which may be described in the [Glossary](Glossary.md).

Major directories
The five major directories are:


 * : The meza application. Only modified when developing for or upgrading meza version
 * : meza configuration. Modified by sysadmins when making configuration changes.
 * : storage of meza information. Modified anytime wiki users do anything on the site.
 * : Files deployed by the meza application. Sysadmins don't need to touch this. Instead, modify  or   and re-deploy.
 * : The webserver root. Deployed just like.

Meza application directory. This should never change except if you upgrade meza (or you're doing development for the meza application itself). This only exists on the controller.


 * : Ansible config file. Ref: http://docs.ansible.com/ansible/intro_configuration.html
 * : base config vars, able to be overridden in /opt/conf-meza/public/public.yml
 * : mostly unused attempts to internationalize meza
 * : extensions installed on all meza servers
 * : CentOS7 VirtualBox Host-Only network interface template
 * : Ref Issue #712: Flatten config/core/ to just config/
 * : Directions on how to contribute to meza
 * : Config file for CodeClimate analysis
 * : Config file for CodeClimate analysis
 * : Template text for new GitHub issues
 * : Template text for new GitHub pull requests
 * : Ref: https://git-scm.com/docs/gitignore
 * : license info for meza
 * : GIF for main README showing usage of meza command
 * : ASCII art of pigeon declaring successful install; Ref Issue #676: reincorporate into deploy
 * (files used for help when using meza command, e.g. )
 * (various files explaining various aspects of meza)
 * : meza's main README
 * (Ansible playbooks. Ref: http://docs.ansible.com/ansible/playbooks.html)
 * (Ansible roles. Ref: http://docs.ansible.com/ansible/playbooks_roles.html)
 * Run on hosts: : Create a Vbox VM; remove when   mature? Or keep for RedHat
 * Setup dev VMs: : Setup host-only network on Vbox VM; Remove after Vagrant? Keep for RedHat setup?
 * Run on controller: : (Used to install   command)
 * Run on controller: : (Entry point for   command)
 * Consolidate :  :
 * Consolidate :  :
 * : Ref Issue #714: Move to tests/integration/ and use in automated testing
 * : Ref Issue #672: determine if still functional
 * : Scripts run on controllers which run several commands to test various functions
 * : Ansible playbook run on controller to setup a test case. creates a fake non-meza source to import from.
 * : Scripts run on a Docker host
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates a generic container
 * : uses init-container to generate a controller container
 * : uses init-container to generate a minion container
 * : Entrypoint script to setup which tests to run. first argument for this script is which test type to run.
 * : finds image URL via JS API, the checks is present on server
 * : several basic tests to verify a meza installation is functioning
 * : checks wiki API, Parsoid, Elasticsearch for a wiki
 * : shim to handle git checkout funnies in Travis
 * : Config file for Travis CI automated testing
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/
 * : Scripts run on controllers which run several commands to test various functions
 * : Ansible playbook run on controller to setup a test case. creates a fake non-meza source to import from.
 * : Scripts run on a Docker host
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates 2 meza Docker containers
 * : generates a generic container
 * : uses init-container to generate a controller container
 * : uses init-container to generate a minion container
 * : Entrypoint script to setup which tests to run. first argument for this script is which test type to run.
 * : finds image URL via JS API, the checks is present on server
 * : several basic tests to verify a meza installation is functioning
 * : checks wiki API, Parsoid, Elasticsearch for a wiki
 * : shim to handle git checkout funnies in Travis
 * : Config file for Travis CI automated testing
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/
 * : several basic tests to verify a meza installation is functioning
 * : checks wiki API, Parsoid, Elasticsearch for a wiki
 * : shim to handle git checkout funnies in Travis
 * : Config file for Travis CI automated testing
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/
 * : Create VMs with Vagrant. Ref: https://www.vagrantup.com/docs/vagrantfile/

These are shell scripts used to create users. Ref Issue #713: Consolidate

/opt/conf-meza


Location of configuration setup for meza. This is split into  and   config. Secret config is intended for sensitive info like passwords. It also houses your  file which declares which servers have which components installed (e.g. Parsoid is installed on  ). The  file may or may not be considered sensitive, but it resides in   nonetheless. Some non-sensitive items may be more convenient to put in. These include things like, which you may only want set to   on your production setup and no others. Since the  file resides in   those environment-specific settings may be better to keep in.

Additionally, the user  is used by meza to perform most actions. Some other actions are performed by the  user. However, we don't create these users in  due to possible conflicts with other user systems. Ref #727.


 * : Use to define extra extensions for your meza installation
 * : MAIN CONFIGURATION VARIABLES FILE!
 * (Any .php files here will be loaded at the end of LocalSettings.php)
 * (Any .php files here will be loaded at the beginning of LocalSettings.php)
 * : this "demo" is known as the wiki ID
 * : favicon
 * : logo
 * ( files here loaded at end of LocalSettings.php for this wiki only)
 * ( files here loaded at beginning of LocalSettings.php for this wiki only)
 * (more wikis with same format as above)
 * : This is the main secret config file. Encrypted by vault password.
 * : AKA "Inventory file", listing server roles. See [Glossary](Glossary.md) for more info
 * : SSL certificate for this environment. Encrypted by vault password
 * : Private key for this environment. Encrypted by vault password
 * (more enviroments if you so choose)
 * : secret key file on controller only. KEEP SAFE!
 * : public key file. This needs to be put on any minion servers.
 * : vault password file for monolith environment. See /opt/conf-meza above. On controller only
 * : (vault password file for other environments)
 * : alternate user for some operations to avoid conflicts
 * : AKA "Inventory file", listing server roles. See [Glossary](Glossary.md) for more info
 * : SSL certificate for this environment. Encrypted by vault password
 * : Private key for this environment. Encrypted by vault password
 * (more enviroments if you so choose)
 * : secret key file on controller only. KEEP SAFE!
 * : public key file. This needs to be put on any minion servers.
 * : vault password file for monolith environment. See /opt/conf-meza above. On controller only
 * : (vault password file for other environments)
 * : alternate user for some operations to avoid conflicts
 * : public key file. This needs to be put on any minion servers.
 * : vault password file for monolith environment. See /opt/conf-meza above. On controller only
 * : (vault password file for other environments)
 * : alternate user for some operations to avoid conflicts

Data storage for meza. This is basically any information generated by using meza, including MariaDB (MySQL) database info, files uploaded by users, logs from meza commands (logs from Apache, HAProxy, system logs, etc are kept in their standard CentOS/RHEL locations), temp files, and backups from running  command.


 * : backups directory populated from running
 * : backups for the "monolith" environment
 * : database backup from specified timestamp
 * : backup of uploads directory
 * : backups for the "production" environment
 * (wikis in production)
 * : files associated with Elasticsearch data
 * : log from automated job runner
 * : log from nightly parsoid restarts
 * : log from this wiki's search index rebuilding
 * : log from this wiki's SMW rebuildData.php
 * : many files and directories associated with MariaDB data
 * : Ref Issue #613: mysql-bin files are not cleaned up by meza currently
 * : temp files which can be deleted. Ref Issue #497: need to keep this directory clean
 * : location where Demo Wiki's uploads/images are kept
 * (more wikis uploads/ directories)
 * : used instead of  if multiple app-servers used. File system distributed between app-servers.
 * : temp files which can be deleted. Ref Issue #497: need to keep this directory clean
 * : location where Demo Wiki's uploads/images are kept
 * (more wikis uploads/ directories)
 * : used instead of  if multiple app-servers used. File system distributed between app-servers.
 * (more wikis uploads/ directories)
 * : used instead of  if multiple app-servers used. File system distributed between app-servers.

This directory is a hidden directory (e.g. starts with a period) because it really shouldn't be modified directly. When the  command is run this directory is set based upon the config set in.


 * : PHP config variable file written based off defaults, secret and public config
 * : Bash config variable file written based off defaults, secret and public config
 * : Extensions to load, written based off core and local extensions
 * : Rebuild Elasticsearch index for a wiki. Deployed by
 * : Wrapper for . Deployed by
 * : Rebuild SMW data for all wikis. Deployed by
 * : A copy of /opt/conf-meza/public, but present on all app-servers, as opposed to  which is only present on the controller
 * : present but not used within .deploy-meza (controller only)
 * : gives app servers access to these files
 * : gives app servers acces to these files
 * : present but not used within .deploy-meza (controller only)
 * : gives app servers access to these logos and wiki specific PHP
 * : Used in cron jobs to run jobs
 * : Used in cron jobs to run jobs

This directory is similar to  in that it is a deployed directory: It is put in place on app servers by the meza application. In fact, it was considered to be placed at  but due to its larger significance as the web root it seemed like it should have its own directory.

build Elasticsearch index for a wiki. Deployed by
 * : entrypoint for landing page
 * : mediawiki application. many items not shown below
 * : where extensions are installed (by the meza application; sysadmins do not need to touch this)
 * : Settings file generated based on your config
 * : Ref Issue #710: backups of LocalSettings.php not required
 * : (see above)
 * : (see above)
 * : logging and performance graphs
 * : server performance
 * : disk space usage
 * : landing page repo. Perhaps should be rolled into meza
 * : directory that symlinks to deployed config, to make logo/favicon web accessible
 * : symlink to  (Ref Issue #709: Security implications)
 * (more wikis)
 * : disk space usage
 * : landing page repo. Perhaps should be rolled into meza
 * : directory that symlinks to deployed config, to make logo/favicon web accessible
 * : symlink to  (Ref Issue #709: Security implications)
 * (more wikis)
 * : directory that symlinks to deployed config, to make logo/favicon web accessible
 * : symlink to  (Ref Issue #709: Security implications)
 * (more wikis)
 * : Wrapper for . Deployed by
 * : Rebuild SMW data for all wikis. Deployed by
 * : A copy of /opt/conf-meza/public, but present on all app-servers, as opposed to  which is only present on the controller
 * : present but not used within .deploy-meza (controller only)
 * : gives app servers access to these files
 * : gives app servers acces to these files
 * : present but not used within .deploy-meza (controller only)
 * : gives app servers access to these logos and wiki specific PHP
 * : Used in cron jobs to run jobs
 * : Used in cron jobs to run jobs

This directory is similar to  in that it is a deployed directory: It is put in place on app servers by the meza application. In fact, it was considered to be placed at  but due to its larger significance as the web root it seemed like it should have its own directory.


 * : entrypoint for landing page
 * : mediawiki application. many items not shown below
 * : where extensions are installed (by the meza application; sysadmins do not need to touch this)
 * : Settings file generated based on your config
 * : Ref Issue #710: backups of LocalSettings.php not required
 * : (see above)
 * : (see above)
 * : logging and performance graphs
 * : server performance
 * : disk space usage
 * : landing page repo. Perhaps should be rolled into meza
 * : directory that symlinks to deployed config, to make logo/favicon web accessible
 * : symlink to  (Ref Issue #709: Security implications)
 * (more wikis)
 * : disk space usage
 * : landing page repo. Perhaps should be rolled into meza
 * : directory that symlinks to deployed config, to make logo/favicon web accessible
 * : symlink to  (Ref Issue #709: Security implications)
 * (more wikis)
 * : directory that symlinks to deployed config, to make logo/favicon web accessible
 * : symlink to  (Ref Issue #709: Security implications)
 * (more wikis)