File:Hadoop and Beyond. An overview of Analytics infrastructure.webm

From mediawiki.org

Hadoop_and_Beyond._An_overview_of_Analytics_infrastructure.webm(WebM audio/video file, VP8/Vorbis, length 23 min 6 s, 640 × 360 pixels, 179 kbps overall, file size: 29.63 MB)

Summary

Description
English: In this tech talk we will be presenting the analytics infrastructure that we have recently rolled out in production. By now probably everybody knows that wikimedia hosts an instance of hadoop from which we are going to extract pageview data in the near future. But .. how exactly does the data get there? We will go over the path that webrequest log data takes from varnish to kafka (a distributed log buffer) to hadoop and the challenges of deploying this java-based infrastructure in production. We will also talk about how can we query the data with hive, an SQL-like interface. How can you set up this stack on vagrant to play with and, last but not least, how we used hive recently to provide GLAM folks with image view stats: Commons:GLAMwiki Toolset Project/NARA analytics pilot
Date
Source https://www.youtube.com/watch?v=tx1pagZOsiM
Author mw:User:Rfarrand (WMF)

Licensing

This video, screenshot or audio excerpt was originally uploaded on YouTube under a CC license.
Their website states: "YouTube allows users to mark their videos with a Creative Commons CC BY license."
To the uploader: You must provide a link (URL) to the original file and the authorship information if available.
w:en:Creative Commons
attribution
This file is licensed under the Creative Commons Attribution 3.0 Unported license.
You are free:
  • to share – to copy, distribute and transmit the work
  • to remix – to adapt the work
Under the following conditions:
  • attribution – You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
This file, which was originally posted to https://www.youtube.com/watch?v=tx1pagZOsiM, was reviewed on 18 July 2016 by reviewer INeverCry, who confirmed that it was available there under the stated license on that date.

Captions

Add a one-line explanation of what this file represents

Items portrayed in this file

depicts

15 July 2014

File history

Click on a date/time to view the file as it appeared at that time.

Date/TimeThumbnailDimensionsUserComment
current23:23, 4 November 201523 min 6 s, 640 × 360 (29.63 MB)LegoktmUser created page with UploadWizard

There are no pages that use this file.