December 2013 - January 2014
Teeveed is a natural language interface for the media center. I built it as my winter break project in 2013.
You talk to it with your smartphone’s web browser. You say “Okay TV, play Game
of Thrones episode 3,” and
teeveed will open that media in VLC on your media
Teeveed is made up of three components:
A coordinator daemon, which runs on your server. It indexes your files for quick search and fuzzy name matching. This process also hosts a webserver with an API, and a remote control client UI.
The remote control client UI. This is a webpage that uses the Chrome Speech APIs to receive user input and pass it on to the coordinator.
The HUD, which displays user intents and matched entities on the big screen. It also opens VLC to play movies. This process controlled by the coordinator.
The intents you see here are parsed from the user’s plain-text input by Wit.ai, then handled by the coordinator daemon.
Development has ceased. I quit working on this early in the Spring 2014 semester as my classes became more demanding.
What went well
I’m very happy with the progress I made on teeveed in the limited time I gave it. I learned a ton.
Leveraging my newly-aquired Java skills from my Winter 2013 classes to build a JavaFX GUI was was fun! I don’t really get the Java hate.
Postgres is really flexible, and has some great extensions for things like full text search.
Chrome has a ton of very cool proprietary APIs, like the speech-to-text API I used for voice.
Wit.ai. I built my app in the early days of the service when the Wit team was making daily improvements. It was fun discovering how powerful the service could be. They also gave great support on Twitter. I’d definitely recommend Wit.ai to a friend.
What went poorly
JRuby. The biggest productivity drain on this project was waiting for JRuby to compile my Ruby code into Java bytecode during app boot. There’s some kind of AOT compilation step you can do to speed this up, but I couldn’t find good documentation for this in 2013. At the time, all the JRuby expertise was locked away inside enterprise users.
As an extension, Maven, the Java build and packaging thingy. Maven is inscrutable to a new user (eg, me), and the slew of plugins I used to build my artifact didn’t help any. Where JRuby was a producitivity drain because it increased my iteration times, Maven setup was just a single massive time-suck.
Complicated architecture. I designed the system to be complex to test my ability to coordinate different components, but I ended up trying to build too much flexibility and configurability. I should have hard-coded more.
Here are my notes from
Sequelfor our ruby-hipster ORM. DataMapper does too much work that keeps us from raw SQL, which we need to implement trigram-based text matching
Use the JavaFX entry point. Right now teeveed boots from
org.jruby.JarBootstrapMain. We need to use a JavaFX main to use JavaFX bundling tools and eliminate dependence on
mvn com.zenjava:javafx-maven-plugin:2.0:fix-classpathfor deployment
Switch from JRuby-based JIT compiling to AOT compiling. teeveed boots really, really slowly, and that’s because there’s a lot of JIT happening when the jar is loaded. Maybe things would go faster if we pre-compiled teeveed (and its gems) somehow.
- Java JDK 1.7
- Maven 2+
- JRuby 1.7.x
- PostgreSQL 9.1+ with the
pg_trgmextension! pg_trgm is one of the ways we find similar-text matches
teeveed uses gem-maven-plugin instead of
Bundler to manage
pom.xml for more information.
git clone https://github.com/justjake/teeveedor similar to get the source
mvn initializewill download and install the required rubygems in
source env.shwill correct your
$RUBY_LIBenvironment variables to point to the gems installed in
teeveed/target/rubygemsand the teeveed sources.
- Hack away. run
teeveed(alias provided by env.sh) to start the daemon, or
clito start the Pry command line.
If things feel too slow,
You should get drip in your ~/bin,
source drip.env.sh before launching any of this ruby junk.
Drip’ll put the spring back in your step!
- Java JDK 1.7
- Maven 2+ (for HUD)
- a Postgres database
The only reason we need Maven and the JDK is because (right now) teeveed isn’t packaged as a JavaFX runtime app. Because the JavaFX runtime does some interesting native loading things, the simplest way is just to move the JavaFX libraries into the implicit system classpath.
I will get around to fixing this, but it’s low priority for my own machines.
(sudo) mvn com.zenjava:javafx-maven-plugin:2.0:fix-classpath
This moves JavaFX onto the classpath so the JavaFX libraries are always loaded. THIS STEP IS ONLY REQUIRED ON USER-INTERFACE MACHINES
make a directory for all of the teeveed resources. I like
example.teeveed.conf.rbinto your teeveed folder.
~/binor somewhere else on your path, and modify
$TEEVEED_HOMEin it to point to your teeveed folder.
- Create a Postgres database and user for teeveed. Make sure your postgres instance is accessable over TCP (unencrypted)
example.teeveed.conf.rbso that it matches your setup. Important things to change:
databasecommand to match the setup of your Postgres database
librarycommand’s path should be ‘/path/to/your/media’
- You’ll need to insert your Wit.ai token for the
- Keep a fast index of all the media on a share for quick search (DONE)
- index needs to respond to natural-language searches (DONE)
- few spelling mistakes, mostly word transpositions (TODO: switch to trigram)
- deleted items should be pruned regularly (DONE)
- delegte music indexing and search to Spotify (TODO)
The index is stored in a PostgeSQL database, which gets us nice
full-text-search for free. Indexing is performed by crawling each
library section on a specified schedule, and adding new items, and
removing old items based on a
At some point we should use the following algorithm when returning search results, so that we never return a file that doesn’t exist:
Natural Language Interface
User story: you sit down on the couch and say, “Ok tv. do we have the new Game of Thrones?”
User story: you want to quickly start music playing while in the kitchen. “Ok tv. Play songs by Royksopp”