Stefan Wienert - Blog

Code, Ruby, China, sonstiges

Ruby pry/irb: fix very slow pasting due to readline on Ubuntu 10.04

| Comments

On an Ubuntu 10.04 LTS Lucid server, I experienced weird behavior of my Ruby shell (pry in this case). Turned out, it's because of a bad version of libreadline.

Building on that Blog article, I tinkered a working solution:

1
2
3
4
5
6
7
8
apt-get remove libreadline6-dev libreadline5
wget https://launchpad.net/\~dns/+archive/test0/+files/libreadline5_5.2-9%7Elucid0_amd64.deb
wget https://launchpad.net/\~dns/+archive/test0/+files/libreadline-gplv2-dev_5.2-9%7Elucid0_amd64.deb\n: 1362009260:0;wget https://launchpad.net/\~dns/+archive/test0/+files/libreadline-gplv2-dev_5.2-9%7Elucid0_amd64.deb
dpkg -i libreadline5_5.2-9~lucid0_amd64.deb libreadline5_5.2-9\~lucid0_amd64.deb
dpkg -i libreadline-gplv2-dev_5.2-9~lucid0_amd64.deb

rvm pgk install readline
rvm reinstall 1.9.3
  • remove system libreadlines
  • download backports from launchpad
  • manually install those via dpkg
  • tell rvm to download and compile readline pkg (I don't know I this was necessary)
  • reinstall Ruby

Afterwards, also make sure, you have no rb-readline in your Gemfile, otherwise this will happen on every "special" keystroke (tab, alt+backspace etc):

1
2
3
4
5
6
7
8
9
10
11
12
[1] pry(main)> Error: invalid byte sequence in UTF-8
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4269:in `block in _rl_dispatch_subseq'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4269:in `each'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4269:in `detect'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4269:in `_rl_dispatch_subseq'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4271:in `_rl_dispatch_subseq'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4251:in `_rl_dispatch'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4669:in `readline_internal_charloop'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4743:in `readline_internal'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/rbreadline.rb:4765:in `readline'
gems/ruby/2.0.0/gems/rb-readline-0.4.2/lib/readline.rb:40:in `readline'
gems/ruby/2.0.0/gems/pry-0.9.11.4/lib/pry/pry_instance.rb:621:in `block in readline'

Faster Rails specs with spring - faster than spork and easier to setup

| Comments

Spring is the new star in the sky of Rails test preloaders/accelerators.

Setup is very easy:

1
2
3
gem install spring
spring binstub rspec
# now there are ./bin/spring and ./bin/rspec

Generating binstubs is not required, but a good idea for even more speedup. You can run specs/console/server/rake with:

1
spring rspec

Running spring the first time, takes the normal bootup time. But afterwards, running the command again is very fast.

Installation issues

On another machine I ran into this issue:

1
2
3
vendor/gems/ruby/1.9.1/gems/spring-0.0.6/lib/spring/sid.rb:1:in `require': cannot load such file -- fiddle (LoadError)
        from vendor/gems/ruby/1.9.1/gems/spring-0.0.6/lib/spring/sid.rb:1:in `<top (required)>'
        from vendor/gems/ruby/1.9.1/gems/spring-0.0.6/lib/spring/env.rb:5:in `require'

I needed to install libffi and recompile, e.g. Ubuntu/rvm:

1
2
apt-get install libffi-dev
rvm reinstall 1.9.3

Guard-rspec

For even more awesomeness, combine it with guard-rspec. The newest versions have spring-support build-in:

1
2
3
4
5
6
# Guardfile
guard 'rspec', :notification=> true,
      :spring => true,
      :binstubs => true do
  #...
end

No more modifications of spec-helper (spork) or recompilation of Ruby (zeus) required! Enjoy your fast specs - and especially generators, rake task etc.

Issues

Important: Remove require 'rspec/autorun' from spec-helper, otherwise, specs will run 2 times, 2nd time controller specs will fail:

1
2
3
4
5
 Failure/Error: get 'welcome'
     NoMethodError:
       undefined method `get' for #<RSpec::Core::ExampleGroup::Nested_4:0x007f64f3713b58>
     # ./spec/controllers/static_pages_controller_spec.rb:7:in `block (2 levels) in <top (required)>'
     # <internal:prelude>:10:in `synchronize'

Fixing Test::Unit::Assertions when using Guard and Spring combined

On one project I had a strange issue, where some dependencies are not found when running guard with spring rspec:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
/profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-rails-2.12.2/lib/rspec/rails/adapters.rb:82:in `assertion_method_names': uninitialized constant Test::Unit::Assertions (NameError)
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-rails-2.12.2/lib/rspec/rails/adapters.rb:88:in `define_assertion_delegators'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-rails-2.12.2/lib/rspec/rails/adapters.rb:108:in `block in <module:TestUnitAssertionAdapter>'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/activesupport-3.2.11/lib/active_support/concern.rb:119:in `class_eval'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/activesupport-3.2.11/lib/active_support/concern.rb:119:in `append_features'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/activesupport-3.2.11/lib/active_support/concern.rb:111:in `include'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/activesupport-3.2.11/lib/active_support/concern.rb:111:in `block in append_features'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/activesupport-3.2.11/lib/active_support/concern.rb:111:in `each'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/activesupport-3.2.11/lib/active_support/concern.rb:111:in `append_features'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/configuration.rb:761:in `include'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/configuration.rb:761:in `safe_include'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/configuration.rb:755:in `block in configure_group'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/configuration.rb:753:in `each'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/configuration.rb:753:in `configure_group'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/world.rb:47:in `configure_group'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:292:in `set_it_up'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:243:in `subclass'
        from /profiles/swi/.rvm/gems/ruby-1.9.3-p374/gems/rspec-core-2.12.2/lib/rspec/core/example_group.rb:230:in `describe'

Fixing: Do not preload spec_helper, only environment. Empty preload indicates only loading of Rails environment, which still gives a significant boost.

1
2
# config/spring.rb
Spring::Commands::RSpec.preloads = []

Learning with Memrise - how I improve my Chinese and geography

| Comments

Several months ago, the webapp memrise.com left beta and went public. Memrise is a platform for learning facts in form of online courses. Great about that is, there is limitation in scope, you can learn:

  • a lot of languages (as of Chinese, Spain, English, German, ....)
  • geography - maps, capital cities, provinces
  • Recognition of famous paintings or music pieces
  • historical facts
  • .. much more

So, in contrast to learning websites that only favor one particular language, here you can mix your curriculum.

Gamification

Memrise presents you the facts in a nice way and often, several users already contributed "Mems" - which is Memrise's term for a mnemonic hook. Later, you are tested on the facts and get points for correctly remembering facts. I really like the interface for answering, which can be 100% keyboard controlled. This way, it really feels more like playing a game.

That points are only for comparing your weekly/monthly activity with your friend's on the highscore. This can be really a motivation to get started again.

Another aspect is that you can create your own private or public courses in a blink. Like, I created some courses for learning Chinese based on my text book and a audio-only course of a regional Chinese dialect.

So, if you ever wanted to learn Spanish, or refresh your school French, that's a nice and fun way to help with vocabulary learning.

Add me as mempal!

Pictures

Some of my courses:

User interface of the test - notice the time counter. It's also possible to include audio facts (recognize words from hearing):

Funny job-titles as click-bait - Why not?

| Comments

Scrolling through most (IT) job boards, there is a unisono of almost the same job-titles, a la: "senior developer $LANGUAGE $FRAMEWORK", which is kinda boring.

Fortunatly, there are some companies that overcome the temptation to join that old story and try something new. This unique job titles are guranteed to catch a lot of attention. Here are some examples from the Stackoverflow Job board:

  • Ridiculously Talented Ruby Developer
  • Want to build the automation framework of your dreams? SDETs wanted
  • Kick Ass PHP Entwickler zur Festanstellung (w/m)
  • Software Developer For a Very Fun Very Successful Startup in the Heart of NYC
  • Magician iOS developer (500.co Startup)
  • Ruby on Rails Extraordinaire
  • Creator of Interesting Things (a.k.a. Senior Rails Developer)
  • Engineering Omnivores - Full Stack
  • Developer who loves great food
  • Ultimate Agile Development Job
  • iOS/Android developer Junior/middleweight
  • Are you a Software Engineer with passion for Open Source? We want you!
  • Why do Server Side Java Engineers wear glasses...?
  • 1337 .NET Developer
  • Ninja Programmer
  • Mostly Awesome Software Engineer
  • Surprisingly Simple Software Developer
  • Everything Web Engineer Extraordinaire (EWEE)

I really like the Everything Web Engineer Extraordinaire :)

Ah, and an old classic, the Code Warrior, experiences with duct tape and WD-40. :)

Dear HR, if your company's culture fits that kind of job title, plz! There is no excuse for yet another "Software developer".

Bewerbung mit XING / Linkedin Lebenslauf

| Comments

Erstmalig veröffentlicht im pludoni developer-Blog

Vor ein paar Tagen haben wir ein neues Feature am Empfehlungsbund freigeschaltet: Bewerbung und Lebenslaufgenerierung mit sozialen Netzwerken, namentlich Xing und Linkedin.

Dabei können sich Bewerber bei der Bewerbung auf Stellen aus ITsax.de, ITmitte.de, MINTsax.de, ... mit ihrem Xing/Linkedin-Profil bewerben. Nach Zustimmung generieren wir on-the-fly einen Lebenslauf und füllen das Bewerbungsformular mit den Stammdaten aus.

Das Ganze ist auch ohne eine Bewerbung möglich. Unter empfehlungsbund.de/cv kann mit einem Klick der eigene Lebenslauf angezeigt werden. (Dabei speichern wir ausdrücklich keinerlei Daten des Lebenslaufs – erst Recht nicht das PDF selbst).

Warum das Ganze?

Wer kennt das nicht? In der Bewerbungsphase ist man damit beschäftigt, seinen Lebenslauf auf den neusten Stand zu bringen, möglicherweise das Ganze noch bei XING/Linkedin zu hinterlegen und schlussendlich individuelle Anschreiben zu erstellen.

Immer mehr Leute haben einen exzellent gepflegte Profile auf XING/Linkedin. Warum also die Daten nur da lassen? Bevor man anfängt nun per Copy-und-Paste das Ganze in einen eigenen Lebenslauf zu gießen, wäre es da nicht schöner, nur mit einem Knopfdruck den Lebenslauf zu generieren? Wir denke ja, mal sehen was die Bewerber so denken :).

Umsetzung

Die Bewerbung wird durch den Empfehlungsbund, eine Ruby-on-Rails-Anwendung realisiert. Darum konnten wir die erstklassige Rails-Bibliothek Omniauth, mit xing und linkedin-Plugins, benutzen. Man muss den Plugins nur die anzufordernden Rechte und ggf. zu holende Daten mit übergeben:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# config/initializers/omniauth.rb
Rails.application.config.middleware.use OmniAuth::Builder do
  provider :linkedin, "apikey", "secret",
    scope: "r_basicprofile r_fullprofile r_contactinfo r_emailaddress",
    fields: %w[formatted-name associations certifications courses date-of-birth
      educations email-address honors twitter-accounts interests headline
      languages:(language,proficiency)
      volunteer:(volunteer-experiences:(role,organization,cause,description,start-date,end-date))
      main-address member-url-resources
      patents:(title,summary,date,url)
      phone-numbers picture-url positions
      publications:(title,date,summary,publisher,url)
      recommendations-received:(recommendation-type,recommendation-text,recommender:(formatted-name,headline,picture-url))
      public-profile-url skills specialties
      summary picture-urls::(original)
  ]
  provider :xing, "apikey", "secret"
end

Wie man oben sehen kann, muss man beim Linkedin-Plugin mit angeben, welche Felder man benötigt. Insbesondere volunteer und recommendations bedurfte einiger Try-and-error's, da die Dokumentation in einigen Stellen vage blieb.

Xing war da deutlich pflegeleichter, allerdings ist hier die Anmeldung einer App komplizierter, da sie bei XING einen Freischaltprozess inklusive Zusendung eines Tokens per Post beinhaltet.

Beide Plugins senden nun Ihre Daten an eine gemeinsame URL, in dem ein Adapter die verschiedenen Profilfelder in ein einheitliches Format übersetzt. Danach wird mithilfe von WickedPDF und wkhtmltopdf ein PDF generiert.

Ausblick

Dank Omniauth können natürlich auch weitere soziale Netzwerke angebunden werden. Wir hatten das Ganze auch für Facebook probiert. Allerdings gibt es (für Normalnutzer) keine Möglichkeit, die Telefonnummer oder Adresse aus der API abzufragen. Damit ist das Ganze für eine Bewerbung relativ witzlos.

Außer Linkedin und XING fielen uns auch keine (im deutschen Markt agierende) soziale Netzwerke ein, die genügend Daten für einen Lebenslauf erfassen. Vielleicht wird es in ferner Zukunft ja mal einen Weibo-CV geben ... :)

Unser Fazit

Insgesamt sind wir recht zufrieden mit dem aktuellen Lebenslauf-Export. Trotz stellenweise sehr unterschiedlichen Formaten wird ein einheitliches PDF generiert. Wenn man das Ganze während einer Bewerbung verwendet, so werden ebenfalls die Stammdaten ausgefüllt.

Jetzt bleibt nur abzuwarten, wie das Ganze bei Bewerbern und Personalern ankommt. Hauptziel ist natürlich eine Verbesserung der "Application-Experience" ;).

current rvm/rubygems problem: Rubygems 1.8.25

| Comments

I just wanted to install newest Ruby via RVM, but got this error:

1
2
3
4
5
6
ruby-1.9.3-p374 - #configuring
ruby-1.9.3-p374 - #compiling
ruby-1.9.3-p374 - #installing
Retrieving rubygems-1.8.25
There is no checksum for 'http://production.cf.rubygems.org/rubygems/rubygems-1.8.25.tgz' or 'rubygems-1.8.25.tgz', it's not possible to validate it.
If you wish to continue with unverified download add '--verify-downloads 1' after the command.

Even rerun with --verify-downloads 1 didn't work, because http://production.cf.rubygems.org/rubygems/rubygems-1.8.25.tgz is simply down. Until this mirror has the version supplied, one can use:

1
2
3
rvm_rubygems_version=1.8.24 rvm install ruby
# or
rvm_rubygems_version=1.8.24 rvm reinstall ruby-1.9.3-p374

Found on twitter:

Practical Ruby: Acquiring QNAP NAS Usage stats

| Comments

The network storage systems of QNAP (like TS-4XX) have a logging function, which can be enabled via the interface. Then, every file access is logged to a SQLite database, located in /mnt/HDA_ROOT/.logs/conn.log on the NAS filesystem (only accessible via SSH).

Using a part of Ruby on Rails, ActiveRecord, to query data, makes this very convenient and short in code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# model.rb
raise "NeedRuby1.9.3" if RUBY_VERSION < "1.9.3"
require "sqlite3"
require "active_record"

class QLog < ActiveRecord::Base
  self.table_name = "NASLOG_CONN"
  self.primary_key = "conn_id"

  # all read on real files
  scope :read, -> { where("conn_action = 2 and conn_res != '---'") }

  def self.from_user(name)
    where(conn_user: name)
  end
end

# "nas" is here a SSH-Alias for the ... you guess NAS
`scp nas:/mnt/HDA_ROOT/.logs/conn.log /tmp/conn.log`
ActiveRecord::Base.establish_connection adapter: "sqlite3",
  :database => "/tmp/conn.log"

Now, this file can be loaded into an IRB/Pry session to play with the data, e.g.:

  • Printing the last 5 open files:
1
puts QLog.read[-5..-1].map{|i|i["conn_res"]}
  • Showing, which user accessed how many files:
1
puts QLog.read.group(:conn_user).count
  • Showing the top 10 popular files:
1
puts QLog.read.group(:conn_res).order("count_all desc").limit(10).count
  • Top 10 popular files of a specific user:
1
puts QLog.from_user("someguest").read.group(:conn_res).order("count_all desc").limit(10).count

and so on.

Programmier-Rätsel als Bewerbung

| Comments

Gerade erreichte mich per XING ein Jobangebot. Etwas anderes dieses mal, ist es doch eine kleine Challenge:

1
2
3
4
5
6
7
8
9
10
11
12
Hi Stefan!

Löse das Rätsel – dann schick uns deinen CV!

Tipp: Die Lösung fängt mit "ZWQu bmVs aGF6 cmFi" an und alle Strings im Array werden genau einmal verwendet.


$a = array(array('ZWQu', 'bmVs', 'd3Vv'),
array('dG5h', 'QGVt', 'aGF6'),
array('eQ==', 'cmFi') );

Interessiert? Mehr Infos gibt es auf XXXX

Interessante Idee, auch wenn mich die Stelle (PHP-Entwickler) nicht wirklich interessiert.

Die Lösung ließ mit einen kurzen Ruby-Skript nicht lange auf sich warten :).

Hacker-Rezept: Chin. Tomaten Rührei

| Comments

Ein recht bekanntes und sehr leicht zuzubereitendes chinesisches Gericht ist "Fan qie chao dan", Tomaten mit Rührei. Dazu sind nur die folgenden Zutaten notwendig:

Zutaten

  • 2 Eier je Person
  • 3 Tomaten je Person
  • Öl zum Braten

Optional noch:

  • Kräuter
  • Beilage, z.B. Reis

Kochgeschirr

  • Pfanne
  • Pfannenwender
  • Schüssel

Vorgehen

  1. Eier in einer Schüssel zerrühren, Salz dazugeben
  2. Tomaten kleinschneiden. Genaues Muster ist unerheblich, da sie sowieso zerkocht werden
  3. Pfanne mit Öl erhitzen (ca. 1-2 Esslöffel)
  4. Wenn heiß, dann Eier reingießen.
    1. In kurzer Zeit sollte das Ei stocken (weiß + hart werden)
    2. Mit einem Holzgerät (a.k.a. Pfannenwender) das Ei schön kleinteilig gestalten
    3. nach max. ca. 5 Minuten sollte sich das Ei langsam verfärben und anfangen anzubraten
    4. dann rausnehmen (wieder in die Schüssel aus 1. geben)
  5. In dieselbe Pfanne wieder etwas Öl gießen
  6. geschnittene Tomaten rein
    1. Diese sollten nach wenigen Minuten zerkochen, also pampig werden
    2. zerkocht: Herd aus, Eier rein
  7. Finalisierung
    1. Wenn Kräuter da, Kräuter rein
    2. Probieren, mglw. noch Salz rein

Schmeckt wunderbar zu Reis, kann man aber auch als einzelnes Gericht ("Snack") genießen. Überrascht hatte mich bei diesem Gericht, wie wenig Zutaten und Kochgeschirr notwendig sind, um ein gutes Gericht zu zaubern. Hier stellen nicht irgendwelche fancy Gewürze oder Geschmacksverstärker die Hauptrolle, sondern das Ei. Probier es aus!

Hacker-Rezepte Eine neue Rubrik, in der ich einfach-zuzubereitende aber trotzdem leckere und gesunde Speisen vorstellen möchte. Das Ganze in einer (hoffentlich) sortierten logischen Aufbereitung :)

Making a torrent mirror for bitlove to support legal podcast torrenting

| Comments

Bitlove is a nice FOSS service, for providing an interface for downloading a lot of podcasts/vodcasts. Having a server which do nothing almost always of the time, I had the idea to mirror some of the torrents.

Deluge is the most popular server-client for bittorrenting. Installation on Ubuntu/Debian is easy, unfortunatly my Ubuntu 10.03 only included a very old version. So setting up a custom ppa:

1
2
3
4
5
6
vim /etc/apt/sources.list

# deb http://ppa.launchpad.net/deluge-team/ppa/ubuntu jaunty main 

apt-get update
apt-get install deluged deluge-console deluge-web

start deluged und deluge-web

1
2
3
4
5
deluged
# returns, starts the daemon

deluge-web
# stays open

Now you can open the UI on your browser: http://host:8112. The default password is "deluge". You can change this under settings. By the way, also change the download directory to something more useful than your home directory. Also increase the number of Active Torrents.

Single torrents can be added via the UI very easily.

Bulk adding torrents

Now, deluge-console can be used to add torrents from feeds. Without a plugin, deluge cannot understand feeds (There is a plugin, but I did not found good installation instructions). So I used ruby to filter the feed and add each item to deluge:

1
2
3
4
5
6
7
8
9
# maybe have to install ruby nokogiri before
apt-get install rubygems1.9
gem1.9 install nokogiri

curl -k --silent https://bitlove.org/c3d2/ds12/feed |  \
  ruby -e 'require "nokogiri";puts Nokogiri.XML(STDIN).search("enclosure").map{|i|i["url"].gsub("https","http") rescue nil}.compact' |  \
  xargs deluge-console add
# download the feed, search for xml-nodes enclosure, get url-attribute and
# replace https to http (because deluge can't handle https

Done! Now keep watching the grass on the webui :) Potentially, add a startup script to automatically start deluged and deluge-web:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# /etc/init/deluge.conf                                                                                                                                               [root@h2039658]
start on (filesystem and networking) or runlevel [2345]
stop on runlevel [016]

exec /usr/bin/deluged -d -c /home/user/.config/deluge

# /etc/init/deluge-web.conf                                                                                                                                           [root@h2039658]
start on started deluge
stop on stopping deluge

exec /usr/bin/deluge-web -c /home/user/.config/deluge


# usage
start deluge