Documentation/Toolkit/Collection audit/Assess

This is Step 4 in the Documentation collection audit process. In Step 3, you narrowed your focus to the most important documents. In this step, you'll assess those docs to determine what types of changes they need.

Assess information overload

For each of the pages in your tracking document, skim the content and make an assessment for the following data points. (Columns in the tracking doc template align with each of the following criteria.)

Audience focus

• Who is the audience or type of user this documentation should help? For example, your audience might be "MediaWiki developers" or "new bot developers". Explore Documentation/Audiences_for_Wikimedia_technical_documentation to find out about common audiences for Wikimedia technical documentation.
• Is all content on the page appropriate for the intended audience? (yes/no)

1. For pages that contain content not appropriate for the intended audience: either move that content, redirect the entire page, or archive it. Don't combine documentation for different audiences.
2. At the beginning of each of your docs, declare who the intended audience is, and direct other audiences away to the content that is made for them. For example, "This page is for any/all SRE team members. For content specific to DC Operations sub-team, visit (some page)."

Content duplication

1. Is some of the content covered elsewhere? If you're not sure based on your initial survey, check this for specific pieces of content by searching across multiple wikis (Wikitech, Meta, MediaWiki), https://doc.wikimedia.org/, and Github/Gerrit/Gitlab.
2. If you find overlapping content: in your tracking doc, note your impression about whether the duplicate content be included in OR removed from your docs. Don't overthink this, just give your initial reaction, taking into account the user tasks and goals of your documentation as identified in Step 1.

Example: if a user's goal is "create a bot", they may need to understand a large range of concepts, like MediaWiki APIs, Wikitext and the structure of wiki pages, bot accounts, and coding conventions. A tutorial about how to create a bot should not cover all those concepts, because they are relevant for many topics beyond just bot development. Instead, a bot tutorial should link to documentation that covers those topics, and clearly state at the beginning that understanding them is a prerequisite for completing the tutorial.

Page length and outgoing links

1. Use the "Page Information" link under Tools in the standard MediaWiki menu to view the page's length in bytes. For pages that are mostly text, a byte length over 20,000 means the page is likely too long and should be split up.
2. In Step 2, when you surveyed linked pages, you probably started to notice some unnecessary links between your docs and other docs. For each doc in your list, skim the content and note if the outgoing links seem useful and relevant to helping your audience complete their task, or if they are superfluous and only serve as distractions.
   1. Remove links that only provide additional or "nice to know" information.
   2. Do not provide a long list of "See also" or "Further reference" links. Technical documentation should not be encyclopedic like Wikipedia. Every link on the page should be relevant and useful to the task the reader is trying to accomplish, or the knowledge they need to do that. Everything else harms their ability to identify that key information.

Assess information gaps

Content freshness

1. At first glance, how correct and accurate is the content on a scale of 1 (fresh) to 5 (completely outdated). Note: last revision date doesn't correlate directly with freshness.
2. Do the names of components or technologies used in the doc reflect the current reality, or have the names (or dependencies) evolved?
3. If score is 3 or above, is your first instinct that most of the content should be: deleted OR updated? (Again, don't overthink this, just give your initial reaction.)

Missing content

Assessing whether content is missing requires subject expertise and familiarity with the current state of the technology. Investigate the following to determine the scope of potentially missing content:

• Are there any features that are undocumented?
• Are there requests for clarification or additional content on Talk pages or in Phabricator tasks?
• Is it clear who is the maintainer of these docs, or where readers should ask questions about the content? Indicate whether you expect doc requests to come through Talk pages, Phabricator tasks, or other mechanisms.
• Are you including the necessary information for the full range of technical expertise or understanding that your audience may have or not have? If you look at a doc and imagine that you know nothing about the topic, consider:
  • What upstream technology, processes, or systems should a reader be aware of before they can understand or use your docs?
  • What technical, social, or other dependencies impact your reader's ability to use the information in your docs? For example: must they have certain software installed, or must they have access to specific systems?

Either indicate prerequisite knowledge or steps at the beginning of the doc, and link to where the reader can go learn them, or make your page a subpage of a more general document where the reader can back up to go learn what they need to know. Remember that a reader may land on this page without context, so your doc needs to direct them to that if they can't proceed without it.

As you explore this, you may want to start filing Phabricator tasks to track issues like this to cleanup when you're done with your initial doc survey. While it's important to understand and improve connections between your docs and other docs, it's easy to get overwhelmed. Stay calm, remember the goals of your documentation, and focus on what your readers need to know to complete their key tasks.

Assess maintenance status

The steps in this section help you understand the problems others have faced when trying to work on these docs. Reviewing past and current maintenance efforts helps you identify pitfalls and avoid redoing work that others have already considered.

Phabricator tasks

• Review the last 20 active Phabricator tasks that mention your key pages and/or your topic.
  • Reviewing by general topic can help you identify high priority content to improve or work on.
  • Reading the task history can help you identify how or why the content is difficult to change, manage, or fix.
• Review the last 10 Phabricator tasks that were closed. How were they closed? Tasks closed without a fix might indicate some problems in the project.

Revisions

Starting with the most important docs in your collection, and proceeding through as many pages as possible, assess:

• When was the last edit made? If it was long ago, is it because content can be considered stable, or abandoned?
• How often do edits typically happen? Did the regular editors stop maintaining content for some reason?

This needs to be evaluated in context. When the project is stable, there is no longer any need to update its documentation. If a given project is actively developed, documentation might become outdated far quicker.

Maintainer outreach

If a page has notable recent activity like Talk page posts or significant recent edits, you should reach out to those people to understand their connection to the content. You may also want to contact the page creator. They may have ideas for how to improve the content, and they may be willing to help with your doc improvement work. If you think content should be moved or deprecated, it's a good practice to mention that to current or recent maintainers before you make such major changes.

Next steps

Sort and filter your tracking document to identify the priority pages to improve for a given focus area. For example, focus on removing duplicate information or on improving the freshness of your docs.

You may want to create Phabricator tasks to indicate the work needed for your list of docs, or you can use other mechanisms like posting on Discussion pages, using a tracking spreadsheet, or editing the pages directly to indicate the work needed.

Phabricator is often the easiest way to coordinate tasks across communities and timezones. If you identified current doc maintainers or stakeholders you should talk to before you make major doc changes, use Phabricator for your task tracking.

Start fixing docs!

• Delete or archive pages that you identified as obsolete.
• Move content that isn't appropriate for your audience into another location. For example, content related to Wikimedia technical infrastructure should generally reside on Wikitech.
• Reconcile duplicate content.
• Make long pages shorter, and don't be afraid to delete extra information that doesn't help your reader understand crucial information or complete a task.
• Use the Documentation Toolkit checklists to review and improve individual pages.