Reading/Web/Projects/A frontend powered by Parsoid/Notes

Notes of the research project through the weeks, and the planning sessions.

Things left to tackle

 * loot-ui & loot
 * Cache invalidation strategy (on loot api and loot-ui client)
 * Reports
 * Html
 * Wikipedia parser analysis (analogous to Restbase analysis)
 * Webpagetest
 * Define how to (and do) measure
 * Slides

Dec 21 - 25

 * ✅ Fix a couple of regressions with cached site and broken lazy loaded images
 * Expand report to include other transformations like superficial markup and ids
 * ✅ Defer loading of initial web app data (to minimize initial HTML payload)
 * Lazy load references in loot-ui
 * Run Webpagetest to generate reports in browser rendering of wp web, wp mobile web, restbase and loot-ui

Dec 14 - 18

 * ✅ Add CI running tests & lint on master & pull requests
 * ✅ Handle 404s and link tag redirects from restbase
 * ✅ Show last modified bar on articles.
 * ✅ Gather data about HTML size with different transformations individually and make report
 * Add tests for UI components (server side testing of browser components and its interactions) (delayed)

Dec 7 - 11

 * Section collapsing (Delayed)
 * References popup (Delayed)
 * ✅ Add tests to loot-ui (reducers, components, actions)

Nov 30 - Dec 4

 * ✅ Add tests to loot-ui (reducers, components, actions)
 * ✅ Implement initial service worker caching (Pull request)
 * ✅ Finish search feature
 * If time, section collapsing

Nov 23 - 27

 * Research Service worker & learn (Reading list)
 * ✅ Document current architecture and code structure in loot-ui (link to docs)
 * ✅ Create application Chrome.
 * ✅ Start adding search feature.

Nov 16 - 20

 * Client side data caching
 * Use indexedDB + fallbacks and cache in the client the state and fetched data to avoid future round trips.
 * Visit an article, click a link. Navigate back. Refresh the page as if you visited later, click on the same link, load should be instant.
 * Lazy loading images
 * Improve transformation of images markup in loot.
 * Lazy load images in viewport in loot-ui.
 * Visit an article, you should see images in viewport loaded. As you scroll you'll see the rest of the images appear instead of all at once blocking the browser and using bandwidth.
 * Sync with Services team
 * Chat about Apps - Api driven frontend - Loot
 * Shared basic services composed in aggregation services.

Nov 9 - 13

 * Web application experience : Deployed version
 * Web application scaffold (routing, state management, view layer, styles & asset pipeline)
 * Webpack + React + Redux + React-Router
 * Move Server Side rendering to web app infrastructure ^
 * Pick up navigation & rendering in client after first render
 * Basic caching infrastructure (assets & api requests)
 * Structured article output from API
 * Move from basic content HTML response to richer JSON structure with content + other entities (references, images, etc)
 * Sync with Services team & Bernd about parsoid html findings
 * Sync with Bernd Notes (12 Nov)

Nov 2 - 6

 * HTML only experience https://reading-web-research.wmflabs.org/wiki/Struggle_session
 * UI urls: https://github.com/joakin/loot/issues/12
 * /wiki/:title
 * Chrome + slim
 * /wiki/full/:title
 * Chrome + full
 * Api:
 * Main transformations
 * Sectioning
 * data-mw
 * comments
 * img -> link
 * style attrs:
 * whitespace: nowrap
 * width: XXX ???
 * strip element from head/only serve body contents
 * Rewrite links (parsoid outpus "./:title")
 * /api/slim/lead/:title
 * Infobox
 * Lead section
 * With a "Read more..." button
 * References (only lead)
 * /api/slim/:title
 * Infobox
 * All sections
 * All references


 * Connect with Peter for performance measuring
 * Sync with mobileapps content service devs.

Oct 26 - 30

 * Clean up loot server produced in offsite's hackathon.
 * Move to more efficient parsing library (libxmljs).
 * Add more transforms than the basic ones.
 * Add basic sectioning.
 * Reproduce research and produce output report.
 * Deploy loot to vm in wmflabs and set up git deployments.
 * Plan next steps, html only version.

Oct 19 - 23
Offsite
 * Hacked a middleman node server to test parsoid html performance with & without stripping content.
 * Anecdotal results from hacking time show very interesting data.
 * Plan to formalize the server and write up a more informed report.

Oct 12 - 16

 * Adapt existing web app prototype to use parsoid html and review possible problems
 * Adapted https://github.com/joakin/webkipedia to use parsoid html from restbase.
 * Serious performance problems
 * Huge html, blocks mobile phones when loading
 * No sectioning, can't get lead sections
 * Live version at http://chimeces.com/webkipedia/
 * Plan to test standalone parsoid html to get performance results next week.