the evolving ultrasaurus » code

getting started with minecraft modding

Sarah — Mon, 01 Apr 2013 01:41:11 +0000

I started learning about creating a Minecraft mod today, using this excellent tutorial (thanks @0×17h and @adudney).

Background: Minecraft is a very popular world-building game. They announced last November that they will release an API, but they have a friendly attitude toward users who have reverse-engineered how to create “mods” (extensions to the game and changes to the behavior of objects in the world). Minecraft Forge is the de-facto standard API toolkit for making mods.

We found that the easiest way to run a server is to run it locally on an old MacBook, allowing external connections to connect to a noip domain and tunnel into our home network.

Prerequisites:We’re running Mac OSX SnowLeopard. We need Java and the JDK, 1.6 or better.

Check that java is installed

$ java -version
java version "1.6.0_37"

Check that the JDK is installed

$ javac -version
javac 1.6.0_37

We installed 64-bit version of Eclipse.

Other than those platform-specific details that I had to look up, we had no problem following the basic modding tutorial. The mod doesn’t do anything yet, but we can run minecraft and see it load:

d3.js experiments in the console

Sarah — Fri, 30 Mar 2012 20:39:44 +0000

d3 (aka Data-Driven Documents) is a great little JavaScript framework for data visualization. It’s got a nice declarative syntax for DOM manipulation that’s quite readable, but takes a bit of effort to understand exactly what it’s doing.

Favorite links:

UPDATE: Dashing D3.js is an amazing series of tutorials with great conceptual grounding
d3 tutorials provide a great conceptual foundation
Thinking with Joins by d3 creator, Mike Bostick, helps explain the syntax for chaining methods
Scott Murray’s d3 tutorial offers a very nice step-by-step, covering a lot of the same ground as my little tutorial below with excellent discussions of the fundamentals.

I like to understand stuff by playing with it interactively, so I created a skeleton index.html which just includes d3.js and a style a div where I’ll display some data.

UPDATE: blank file below posted here


  
    d3 experiment
                src="https://raw.github.com/mbostock/d3/master/d3.js">
    
    
      .box {
        background-color: skyblue;
        width: 24px;
        height: 18px;
        padding: 4px;
        margin: 1px;
      }

Then in the FireBug console, we can interact with d3, the top-level object that allows us to access all of d3’s goodness.

>>> d3
Object { version="3.0.1", random={...}, ns={...}, more...}
>>>  body = d3.select("body")
[[body]]

Like jQuery, d3 let’s us “select” one or more DOM elements to operate on them. I only have one body tag, so I just get one element in an array — not yet sure why it needs a nested array. Now I can manipulate the DOM:

>>>  body.append('p').text('Hello d3!')
[[p]]

and “Hello d3!” appears at the top of my page. Yay! Of course that could have been written in a single line like:

d3.select("body").append('p').text('Hello d3!')

and if I want to change the text, I can use a regular old css selector to grab the paragraph element I just created:

d3.select("body p").text("Welcome to d3")

or, using the reference to the ‘body’ variable I created above:

body.select("p").text("d3 is cool")

Data-driven Boxes

Ok, now that we understand the basics, let’s put some boxes on the page:

body.append('div').attr('class','box')

and let’s add a couple with text in them:

body.append('div').attr('class','box').text('hi')
body.append('div').attr('class','box').text('foo')

With my set of boxes, I can select one or all of them:

>>> d3.select('.box')
[[div.box]]
>>> d3.selectAll('.box')
[[div.box, div.box, div.box]]

Then I can specify data to bind to each box and display it. I’ve read that d3 can deal with all sorts of data (like json, csv, etc.) but we’ll start with an array of numbers.

>>> my_data = [20, 7, 32]
[20, 7, 32]
>>> d3.selectAll('.box').data(my_data).text( function(d) { return d } )
[[div.box, div.box, div.box]]

We can see that our data is associated with the DOM element and we can get at it via JavaScript in the console. (Of course, we should only do that for debugging. I would guess that __data__ is the private implementation of d3’s data binding.)

>>> d3.select('.box')[0][0].__data__
20

We can change the data like this:

>>> new_data = [10, 50, 25]
[10, 50, 25]
>>> d3.selectAll('.box').data(new_data)

You’ll see that the page doesn’t change visually, but in the console, you can see that the data does:

We need to explicitly tell d3 to do something with the data like this:

d3.selectAll('.box').text( function(d) { return d } )

We can also use this handy shortcut:

d3.selectAll('.box').text( String )

fixing brew install opencv on osx

Sarah — Fri, 30 Mar 2012 11:46:27 +0000

This is more about fixing my brew install, than about opencv. As with many install issues the root cause was actually pretty simple, but finding it was challenging. Along the way, I fixed a number of issues which took a bit of digging to find, so I’m leaving a little trail on the web in case other people run into the same things — or in case some inspired open source citizen has time to add better solution messages to brew. The first step of any solution, is, of course, understanding the problem.

$ brew install opencv
==> Installing opencv dependency: cmake
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/cmake-2.8.7-bottle.tar.gz
######################################################################## 100.0%
Error: SHA1 mismatch
Expected: f218ed64ce6e7a5d3670acdd6a18e5ed95421d1f
Got: 3a57f6f44186e0dba34ef8b8fb4a9047e9e5d8a3

solution:
$ brew update
:
:
Error: Failed executing: make install (libtiff.rb:18)
If `brew doctor’ does not help diagnose the issue, please report the bug:
https://github.com/mxcl/homebrew/wiki/reporting-bugs

tl;dr;
install command-line tools from developer.apple.com

before I figured that out I fixed all of the issues found with ‘brew doctor’

$ brew doctor

Warning: Some directories in /usr/local/share/man aren’t writable.
This can happen if you “sudo make install” software that isn’t managed
by Homebrew. If a brew tries to add locale information to one of these
directories, then the install will fail during the link step.
You should probably `chown` them:

/usr/local/share/man/de
/usr/local/share/man/de/man1

solution:
$ sudo chown sarah /usr/local/share/man/de/*
$ sudo chown sarah /usr/local/share/man/*

Warning: “config” scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.

Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following “config” scripts:

/Library/Frameworks/Python.framework/Versions/2.7/bin/python-config
/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7-config

solution:
Uninstalled python, which I don’t use much — I figure I can install later with brew
$ sudo rm -rf /Library/Frameworks/Python.framework/Versions/2.7
$ sudo rm -rf /Library/Frameworks/Python.framework/Versions/2.7
$ sudo rm -rf “/Applications/Python 2.7″
$ sudo rm /usr/local/bin/py*

Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built.

coreutils
geoip

solution:
$ brew link coreutils
Linking /usr/local/Cellar/coreutils/8.12… 0 symlinks created
$ brew link geoip
Linking /usr/local/Cellar/geoip/1.4.6… 2 symlinks created

Warning: You have uncommitted modifications to Homebrew’s core.
Unless you know what you are doing, you should run:
cd /usr/local && git reset –hard

tried this:
$ cd /usr/local && git reset –hard
HEAD is now at ffb9aa5 Remove “__brew_ps1″ function from completion
–> didn’t work

solution:
$ pushd /usr/local
$ git status
–> lots of untracked files, no idea how I got into that state
$ git add .
$ git reset HEAD –hard
$ popd

Warning: Your Xcode is configured with an invalid path.
You should change it to the correct path. Please note that there is no correct
path at this time if you have *only* installed the Command Line Tools for Xcode.
If your Xcode is pre-4.3 or you installed the whole of Xcode 4.3 then one of
these is (probably) what you want:

sudo xcode-select -switch /Developer
sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer

DO NOT SET / OR EVERYTHING BREAKS!

I don’t have anything at /Developer, so I did this:
$ sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer

$ brew doctor
Your system is raring to brew.

Of course, it wasn’t, the key clue for me was finding this in the long stream of installation output:
tiffgt.c:35:11: fatal error: ‘OpenGL/gl.h’ file not found

which convinced me that I was missing some fundamentals. Searching on the text of the error led me to:
https://github.com/mxcl/homebrew/issues/11088

Ideally ‘brew doctor’ would have caught that I was missing the command-line tools that don’t get installed automatically with XCode 4.3. I installed those and all was well.

cucumber and custom rspec matchers with rails 3.1

Sarah — Sun, 22 Jan 2012 16:40:52 +0000

I’m working my way through an epic Rails 3.1 upgrade and some of my cucumber features were failing because I was using a custom RSpec matcher and the method wasn’t found.

My custom matcher looks something like this:

module CustomMatchers

  class XmlSubsetMatcher
      :
  end

  def be_xml_subset_of(expected)
    XmlSubsetMatcher.new(expected)
  end

and when I ran my feature I was getting this failure:
undefined method `xml_subset_of?' for # (NoMethodError)

As it turns out, in my zeal to make sure everything was using the latest and great new stuff, I had forgotten to move over this critical configuration line in cucumbers env.rb:

World(CustomMatchers)

Now, my cucumber feature is happily failing cuz my code doesn’t work. Whew. I couldn’t find this documented anywhere and I’m not even sure where this documentation would belong. I found a hint on the cucumber wiki rspec expectations page, but none of the code on that page is actually needed when using cucumber with Rails, so I decided not to touch it and just write this blog post.

ffmpeg on osx lion

Sarah — Sun, 08 Jan 2012 05:40:37 +0000

I found that I needed to convert an m4a audio file (which is what QuickTime saves when I record audio) to a wav file, so I decided to use my favorite “can opener.” The versatile open source ffmpeg tool has always seemed to be able to convert anything to anything in audio-video formats.

I decided to pull the source from git:
$ git clone git://source.ffmpeg.org/ffmpeg.git $ cd ffmpeg/

Stable versions are tagged (which I could see with “git tag -l”). I don’t need to live on the edge right now, so I switched to the tag “n0.9.1″ which I assume is for the latest stable build “harmony” 0.9.1 and made a local branch based on that.
$ git co n0.9.1 $ git checkout -b n0.9.1

Instructions for building ffmpeg are in the “INSTALL” file. I discovered I needed yasm, which I could install with brew. Here’s what I did:
$ brew install yasm $ ./configure $ make CC libavdevice/alldevices.o CC libavdevice/avdevice.o CC libavdevice/lavfi.o AR libavdevice/libavdevice.a CC libavfilter/af_aconvert.o libavfilter/af_aconvert.c:53: warning: function declaration isn’t a prototype libavfilter/af_aconvert.c:105: warning: function declaration isn’t a prototype CC libavfilter/af_aformat.o CC libavfilter/af_anull.o CC libavfilter/af_aresample.o : : ffserver.c: In function ‘parse_ffconfig’: ffserver.c:4236: warning: ‘avcodec_get_context_defaults2’ is deprecated (declared at ./libavcodec/avcodec.h:3948) ffserver.c:4237: warning: ‘avcodec_get_context_defaults2’ is deprecated (declared at ./libavcodec/avcodec.h:3948) LD ffserver_g CP ffserver STRIP ffserver
I saw a lot of warnings, but they didn’t seem to negatively affect what I was trying to do. I found a nice blog post from catswhocode to remind me of the usage, and was able to use this simple command:

$ ./ffmpeg -i frog.m4a frog.wav ffmpeg version 0.9.1, Copyright (c) 2000-2012 the FFmpeg developers built on Jan 7 2012 21:19:08 with llvm_gcc 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) configuration: libavutil 51. 32. 0 / 51. 32. 0 libavcodec 53. 42. 4 / 53. 42. 4 libavformat 53. 24. 2 / 53. 24. 2 libavdevice 53. 4. 0 / 53. 4. 0 libavfilter 2. 53. 0 / 2. 53. 0 libswscale 2. 1. 0 / 2. 1. 0 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'frog.m4a': Metadata: major_brand : M4A minor_version : 0 compatible_brands: M4V M4A mp42isom creation_time : 2012-01-08 05:09:05 Duration: 00:00:07.22, start: 0.000000, bitrate: 206 kb/s Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, s16, 201 kb/s Metadata: creation_time : 2012-01-08 05:09:05 handler_name : Output #0, wav, to 'frog.wav': Metadata: major_brand : M4A minor_version : 0 compatible_brands: M4V M4A mp42isom creation_time : 2012-01-08 05:09:05 encoder : Lavf53.24.2 Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s Metadata: creation_time : 2012-01-08 05:09:05 handler_name : Stream mapping: Stream #0:0 -> #0:0 (aac -> pcm_s16le) Press [q] to stop, [?] for help size= 1244kB time=00:00:07.22 bitrate=1411.3kbits/s video:0kB audio:1244kB global headers:0kB muxing overhead 0.003611%

$ ls frog.m4a frog.wav

Success!!

rails 3.0 and rake 0.9.2

Sarah — Sun, 11 Sep 2011 16:37:17 +0000

I really want to upgrade a Rails 3.0 project to Rails 3.1, but I’ve done a few spikes and it lacks test coverage, so I decided to pull in cucumber and write some features before moving forward.

I added cucumber-rails to my gemfile, and ran “bundle” and got this error:

/Users/sarah/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:289:in `load': uninitialized constant Psych::Syck (NameError)

What I really needed was to update my Ruby Gems (bundle update –system) but before I discovered that I did “bundle update” which moved me forward to rake 0.9.2, so I started getting these warnings:

/Users/sarah/.rvm/gems/ruby-1.9.2-p290@pie-bakery/gems/psych-1.2.1/lib/psych.rb:93: warning: already initialized constant VERSION
/Users/sarah/.rvm/gems/ruby-1.9.2-p290@pie-bakery/gems/psych-1.2.1/lib/psych.rb:96: warning: already initialized constant LIBYAML_VERSION
WARNING: Global access to Rake DSL methods is deprecated.  Please include
...  Rake::DSL into classes and modules which use the Rake DSL methods.
WARNING: DSL method Bakery::Application#task called at /Users/sarah/.rvm/gems/ruby-1.9.2-p290@pie-bakery/gems/railties-3.0.0/lib/rails/application.rb:214:in `initialize_tasks'

So, I’ve learned from google, stackoverflow, various blogs and my twitter friend @excid3 that I need to update my Rakefile to include:

require 'rake/dsl_definition'
require 'rake'
include Rake::DSL

That lets me use rake (yay!). I still have the following two warnings:

/Users/sarah/.rvm/gems/ruby-1.9.2-p290@pie-bakery/gems/psych-1.2.1/lib/psych.rb:93: warning: already initialized constant VERSION
/Users/sarah/.rvm/gems/ruby-1.9.2-p290@pie-bakery/gems/psych-1.2.1/lib/psych.rb:96: warning: already initialized constant LIBYAML_VERSION

which I’m hoping will go away with the Rails 3.1 upgrade, but I thought I would write up the rest of it in case it helps other wayward souls on their journey.

what exactly does rake spec do?

Sarah — Sun, 08 May 2011 21:41:35 +0000

$ rake spec
(in /Users/sarah/src/../my_app)
You have 1 pending migrations:
20110416135407 CreateCourses

The rake spec command reminds us that we need to run our migration before running
our tests. In fact, it does a whole lot more than that. There are a whole bunch of best practices rolled in that one
command. To see exactly what is going on, we can run rake spec with the –trace
option:

$ rake spec --trace (in /Users/sarah/src/tfr/svn/Book/code/class_app_new_source) ** Invoke spec (first_time) ** Invoke db:test:prepare (first_time) ** Invoke db:abort_if_pending_migrations (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute db:abort_if_pending_migrations ** Execute db:test:prepare ** Invoke db:test:load (first_time) ** Invoke db:test:purge (first_time) ** Invoke environment ** Execute db:test:purge ** Execute db:test:load ** Invoke db:schema:load (first_time) ** Invoke environment ** Execute db:schema:load ** Execute spec

When it says invoke it is calling a particular rake task, but then it will call its dependencies. To really see what is happening in what order, check out the execute commands. The commands db:test:prepare and db:test:load don’t do much themselves, aside from setting up the environment and executing another task or two. We can see from the output that rake is actually executing the following steps:

Don’t run the specs if there are pending migrations in the development database. (db:abort_if_pending_migrations)
Drop the test database (db:test:purge)
Load the schema into the test database (db:schema:load in environment “test”)

These steps make sure that we are always testing in a clean environment, so we know exactly what we’re testing when we run our specs.

The code that makes this happen in Rails 3, can now be found in railties. (Thanks to @pixeltrix for pointing me to it.)/62206174505873408

repl rspec mocks

Sarah — Tue, 05 Apr 2011 01:59:19 +0000

REPL (Read-Eval-Print-Loop) is a great way to learn. With Ruby, the experience is enabled with irb. Sometimes, to do this we need to peek into the innards of things, which I find to be an extremely effective way to explain mocks and stubs. It’s a regular part of my Ruby curriculum, even though I have needed to figure out the syntax three times in the last couple of years. (Many thanks to Jen-Mei Wu for the most recent iteration.) I still think it is worth it, even though it seems to change with crazy frequency.

Just in case anyone else ever wants to do this with current or previous versions of RSpec, I thought I would write it down before old versions become lost in the mists of time:

RSpec 2.9

[update for repl mocks for RSpec 2.9 by Curtis Schofield]

>> require 'rspec/mocks/standalone'
>>Time.stub(:now).and_return(10,20)
>>Time.now
10
>>Time.now
20
>>Time.now
20

RSpec 2.5

>> require 'rspec/mocks'
>> include RSpec::Mocks::Methods
>>Time.stub(:now).and_return(10,20)
>>Time.now
10
>>Time.now
20
>>Time.now
20

RSpec 2.0

>> require 'rspec/mocks'
>> require 'rspec/mocks/extensions/object'
>>Time.stub(:now).and_return(10,20)
>>Time.now
10
>>Time.now
20
>>Time.now
20

RSpec 1.3

>> require 'spec'
>> require 'spec/mocks'
>>Time.stub(:now).and_return(10,20)
>>Time.now
10
>>Time.now
20
>>Time.now
20

rails 3 vs. rails 2 validation errors

Sarah — Tue, 07 Sep 2010 10:14:06 +0000

Not sure if this is a bug or a feature. I’d guess it is here for a reason, and maybe I’m late for noticing, but Rails 3 errors now provides an array for each attribute, whereas in Rails 2.3 it was just a string. Here’s the output from two almost identical applications…

Loading development environment (Rails 2.3.8)
>> person = Person.new
=> # >> person.valid?
=> false
>> person.errors
=> # >> person.errors[:first_name]
=> “can’t be blank”

Loading development environment (Rails 3.0.0)
>> person = Person.new
=> # >> person.valid?
=> false
>> person.errors
=> {:first_name=>["can't be blank"]}
>> person.errors.class
=> ActiveModel::Errors
>> person.errors[:first_name]
=> ["can't be blank"]

I didn’t see that in the release notes, but it failed my tests for ActiveRecord class. Someone else must have a list of these details, yes?

rails security review checklist

Sarah — Wed, 27 Jan 2010 08:00:12 +0000

I’m reviewing the security of a web app built with Ruby on Rails, so I put together a checklist for a security audit. This isn’t a bank or high security situation, but there were a number of engineers and quite a bit of open source code, so I thought a few checks were in order.

Here’s the list I came up with that I thought other folks might appreciate as a starting point (special thanks to the sfruby list, Mike Gunderloy, and Scott Bronson for feedback):

0) Make sure your Rails and gems are up to date for latest security patches (see rails security mailing list for recent advisory notes)

1) Active Record audit:
  A) SQL injection:
    (i) whole word search for “find”, “first”, and “all” then visually inspect all instances of ActiveRecord find calls for potential SQL injection vulnerability (also search for “sql” not whole work search to find find_by_sql and “execute” to find cases where raw sql is executed.
    (ii) search your models for “named_scope” and check :conditions
  B) check for mass assignment Either disable mass assignment as Eric suggests in his article, or audit its use. If doing an audit, check every model to make sure it declares which attributes are settable with attr_accessible. (While attr_protected may technically work, a white list approach is recommended by security experts and the rails security advisory on this topic)

2) Scripting attack: search all eRB files for <%= and ensure that if dynamically generated text was originally entered by the user, it is HTML escaped. Consider rails_xss

3) Secure Access: If some of the site does not have public access, check controllers and ensure that public actions are specifically allowed and that protected access is the default

4) search for “eval” (whole word) and verify that usages are safe (assume javascript eval is ok)

5) search for “forgery” (not whole word), make sure that
config.action_controller.allow_forgery_protection = false
is only disabled in test config
protect_from_forgery should be in the ApplicationController, unless there is a good reason for it not to be

6) check user auth and review that controller actions are limited to expected use

7) passwords: not saved as clear-text in the db, not logged

8) check that private data is not stored in cookies