User:Ori.livneh

From MediaWiki.org
Jump to: navigation, search

Contents

Redis[edit]

Memory usage of redis suspiciously exceeds size of dump[edit]

  • What types of keys do you have? How big are they? What are they storing?
  • In INFO output: memory fragmentation ratio, used_memory_rss
  • Debug memory use with DEBUG OBJECT <key>

Sets[edit]

  • It's not documented, but you can use ZINTERSTORE and ZUNIONSTORE with SETs in Redis, they act as though their members all have a score of 1.
  • BLPOP will block the connection until a new item is available to pop from a list

Misc[edit]

"Massively scalable counter systems like rainbird are intended for high cardinality data sets with pre-defined hierarchical drill-downs. But they break down when supporting arbitrary drill downs across all dimensions." [1]

Rainbird data model[edit]

struct Event {
   i32 timestamp,
   string category,
   list<string> key,
   i64 value,
   optional set<Property> properties,
   optional map<Property, i64> propertiesWithCounts
}

[2]

Notes[edit]

  1. http://metamarkets.com/2011/druid-part-i-real-time-analytics-at-a-billion-rows-per-second/
  2. http://assets.en.oreilly.com/1/event/55/Realtime%20Analytics%20at%20Twitter%20Presentation.pdf

Clojure[edit]

22:20 (SrPx) ns creates a namespace and uses it, right? mpan
22:20 (SrPx) so it is like a thing you call once?
22:20 shachaf has left ()
22:20 (SrPx) I'm avoiding it because I do not understand. do I have to name it following some convention based on my files or something?
22:21 (mpan) uh, usually it goes like this
22:21 (mpan) you have separate namespaces, one per file, with corresponding names
22:21 (mpan) the ns statement switches the currently-used namespace for evaluating stuff
22:22 (mpan) so if you eval something, it's done so in a given namespace
22:22 (mpan) so at the repl, you can use ns to switch to evaling stuff in the context of a different namespace
22:23 (mpan) http://blog.8thlight.com/colin-jones/2010/12/05/clojure-libs-and-namespaces-require-use-import-and-ns.html this might help
22:23 (mpan) it covers both separating stuff into namespaces and bringing stuff in from them
22:24 eldariof has joined (~CLD@pppoe-222-244-dyn-sr.volgaline.ru)
22:24 statonjr has left IRC (Quit: statonjr)
22:24 (SrPx) ok thank you mpan
22:24 (mpan) you're always in some name space or another
22:24 lggr has joined (~lggr@84-73-159-126.dclient.hispeed.ch)
22:24 (mpan) and you could switch between them with ns calls
22:25 (mpan) the reason for the naming convention and file structure is so that the runtime knows where to look when you want to bring something in