The Magic Bean

I have this theory about "magic beans." Beans, beans the magical fruit -- the more you Magic Beans are problem fixes for those issues that are making you pull your hair out. "Beans" because they're usually really insignificant in 'size' and "Magic" because they're not normally logical, well-known, or obvious.

For instance: I just updated one of my NAS posts below with a magic bean I had forgot about. The longer story being that I had been in the middle of a package update in opensolaris when I realized I hadn't snapshot my root filesystem. Right about then it crashed. Long story short, I had to reinstall the OS and get my networking reconfigured. But try as I might... no matter how exactly I followed all the right recipes on getting networking configured it just wouldn't work. Enter the magic bean. I found a thread that looked familiar, then it hit me... there were these 4 little commands one had to run, and "poof" -- the rest of my configuration started magically working. These commands aren't normal... in fact, they're only necessary because of a defect in my network adapters or driver.

To clarify, typographical errors in code do not count as Magic Beans. They're small and not obvious... but you have the knowledge to fix them if you could see them. Magic Beans are the problems that we can't see (because we don't know how to fix them?).

Contagious Intellectualism

To say this friend of mine is brilliant is an understatement. But alas the English language doesn't let me make superlatives out of almost any word like some others do... so that will have to suffice.
I had known since before I met him that he was smart -- but there is a difference between knowing something and really knowing something. The other day I had included him on an email about a game idea I had been batting around for a while (something like a cross between CSI & Mastermind). His response contained three questions... and they absolutely blew me away. Who thinks this way? Maybe it's just me, but just being able to conceive and ask these questions is a mark of genius in my book.

...This [game] idea is closely related to a couple of things I've been noodling around with lately:
1. How can we identify "signatures" of phenomena we don't yet recognize as important?
2. How can we optimize (in time and cost) forensic interrogation procedures for "mixed" phenomena?
3. How can we model interrogations of poorly understood phenomena?
...

Wow. It feels like I'm only "noodling" around much more mundane things these days. Perhaps now that the ADHD meds are clear of my system the lofty intellectual noodling will start back up.

Pork Loin, Served by Proxy

If anything could peg the Meter-o-Machismo and firmly cement myself in the top Alpha Geek ranks, this would be it...

I want to internet enable my bar-b-que.

The backstory, like all of my stories, is fairly long and complex so we'll suffice it to say that I have wanted to undertake a project such as this for quite some time. After my most recent technical project (the storage server, covered earlier in the blog), I find myself still "jonesing" to do some hybrid creative/technical work. And since my Traeger (best. food. ever.) needs torn apart to fix the auger, I figured it would be a good time to muck about in its electronic innards.

My initial requirements:

• After all modifications it must still function as a smoker/bbq.
• The BBQ will interface with my network via a wireless signal (802.11 b/g).
• The user should be able to set a temperature (in degrees Fahrenheit) via the control panel.
• The temperature control may also be set securely via the network.
• BBQ will monitor one or two temperature probes (one for BBQ temp, one for food temp) and attempt to maintain the set temperature.
• Statistics (temperature settings, temperature measurement history) will be available via the network (and potentially the internet)
• [Optional] Cooking "profiles" (ideal temperature curves over time) may be loaded and run.

Here are some of the components I'm looking at using (not necessarily any or all of them will make the final "cut"):
Processor:
Right now I'm torn between the Arduino or the Propeller. Both have their pros & cons.

Wireless Interface: the Matchport Pro b/g wireless embedded device server -- this thing is pretty cool. It's basically a secure web-server combined with a wireless radio chipset. I should be able to interface to this with whatever processor I choose.

I had been looking at just using a Gumstix system with a wi-fi add-on card, but putting together a system that would do what I wanted seemed to be a bit on the spendy side.

I obviously have a lot more thinking and planning to do, but if I do end up going ahead with this project I will of course let everyone know how it went.

WTF, doesn't Solaris speak Linux? (The NAS Project, part 3)

[covering software installation, configuration, troubleshooting]

When we last left our hero, he had just successfully powered on his newly assembled NAS and was rejoicing. The emotion was short lived.
Prior to ordering everything, I did some searches against the solaris hardware compatibility list and didn't really turn up anything untoward.
I still had some issues, however. By far, the biggest stumbling blocks for me were getting the network adaptors functioning and having the operating system recognize my sata drives.

But I'm getting ahead of myself. The first step was to actually install the OS. I had downloaded the opensolaris 2005.08 iso and burned it to disc. It's a live-cd, so I was able to boot into it, and then run the install. Unfortunately, I must have had some corruptions in the download or the burn, because the install didn't work correctly. I tried again, and this time the download was good (verified the hash check, which I should have done originally - shame on me). But the burn didn't work -- great, another coaster. The third time was the charm, though.
Opensolaris was installed and booting.

Then it freaked out. It would start, but as soon as the bios would post the video would go out. Or the video would go out when I logged into the gui. Or it would start, then the video would go out when the screensaver kicked in, and never come back.

After much hair pulling & a helpful suggestion from the LogicSupply tech support, I tracked the problem down to the memory being in a bad spot. I moved the RAM chip to the other slot, ran a memory test from a bootable disc (http://www.memtest.org/) which showed 100% error free, and everything worked great after that.

The next problem I tackled was getting it to see my network... "N" is the first part of NAS, after all. The network adaptors on my motherboard are Broadcom BCM5787M (vendor id: 14e4, device id: 1693), and not supported natively in opensolaris yet.
After much searching & forum reading and attempting various & sundry things, I stumbled across this thread:
http://opensolaris.org/jive/thread.jspa?messageID=195224
It was during all of this trial and error that I came to realize networking and the various commands in opensolaris were different enough from linux to cause me a decent amount of confusion.

Anyhow, I ended up downloading & installing the archived drivers, and afterward my system says the broadcom devices were "up" -- I just wasn't getting an IP.
Finally, after a week and a half of tweaking & reading through the various forums, I've managed to get everything working & have it persist through restarts.
It still grumbles about configuration errors & forcing things into maintenance mode when I boot but things seem to be working fine.

NOTE: Don't bother with the network gui. Don't even go into it.
NOTE: In opensolaris, your network adaptors are named for their drivers. Since I was using the broadcom driver, my two adaptors are bcme0 and bcme1.
NOTE: hostnames and IP's have been changed to protect the innocent :)
NOTE: the stuff below is mostly from memory. I'll try to double-check it before I post but may not get to it. UPDATE: Double checked, and I think this is complete.

First, I disabled the NWAM service (network auto magic? -- it didn't work, either):
# svcadm disable /network/physical:nwam

I enabled the normal network service:
# svcadm enable /network/physical:default

[EDIT] I forgot to mention the magic bean... the thing that makes all of our networking stuff gel. This is specific to the hardware/drivers, so isn't totally necessary for anyone else not using the bcme.
# svccfg -s network/physical:default setenv DLPI_DEVONLY 1
# svcadm refresh network/physical:default
# svccfg -s network/physical:nwam setenv DLPI_DEVONLY 1
# svcadm refresh network/physical:nwam
# reboot

http://www.opensolaris.org/jive/thread.jspa?threadID=61541&tstart=50

Without doing this, you can have an IP address & snoop the network traffic, but not be able to ping/respond or do anything else.

I put my router's IP in the /etc/defaultrouter file:
# cat /etc/defaultrouter
192.168.0.1

I edited my etc/hosts to add a static IP for the adaptor/box:
# cat /ets/hosts
127.0.0.1 localhost
192.168.0.10 supernas

I added my DNS entries to /etc/resolv.conf (one for my router's IP which should do DNS lookups, as well as one for OpenDNS):
# cat /etc/resolv.conf
nameserver 192.168.0.1
nameserver 208.67.222.222

I edited copied /etc/nsswitch.dns over to /etc/nsswitch.conf and then edited the .conf file so the hosts line read:
# cat /etc/nsswitch.conf | grep hosts
hosts: files dns

I added a default route to my router:
# route -p add default gateway 192.168.0.1 1

I edited /etc/netmasks (for various complicated reasons, I have a couple different subnets at home. Most people will use the default 255.255.255.0):
# cat /etc/netmasks
192.168.0.0 255.255.254.0

I put my system's hostname & some configuration commands in the /etc/hostname.[interface] file:
# cat /etc/hostname.bcme0
supernas netmask + broadcast + up

And that seemed to do it!

The next big problem I had was the fact that opensolaris couldn't see the really cool hot-swap sata drives, no matter how much tweaking I did.
You can find some of my problems in this thread I started (the thread got away from me, but the first few posts are still valid): http://opensolaris.org/jive/thread.jspa?threadID=62493

With the backplane in place & the drives socketed in their bays, opensolaris never recognized them as being anything other than "empty" no matter what bios setting I used (AHCI, legacy, etc.) If I directly connected them to the mobo it recognized them just fine, however. Drat -- I had really wanted to have the hot-swap capability.
Oh well, maybe the chenbro chipset will be supported in a later release of opensolaris.
I decided to just directly connect the drives to the motherboard and removed the backplanes (after some case dissasembly). BTW -- if anyone is developing drivers for this backplane, I'd be willing to temporarily donate one of the backplane boards to the cause, assuming I could get it back once they're supported :)
I then test fit the drive caddys in to see how the cables would fit through the new holes. Nope... had to notch the little metal plate that held the backplanes.
Once that was done, everything fits nicely and opensolaris saw the drives just fine.
I checked the "location" of the new drives:
# cfgadm | grep sata
App_Id Type Receptacle Occupant Condition
sata0/0::dsk/c6t0d0 disk connected configured ok
sata0/1::dsk/c6t1d0 disk connected configured ok
sata0/2::dsk/c6t2d0 disk connected configured ok
sata0/3::dsk/c6t3d0 disk connected configured ok

(or something similar - that's not the exact output)

From that point, creating my 2+ terabyte storage pool was a matter of:
# zpool create tank raidz c6t0d0 c6t1d0 c6t2d0 c6t3d0

Then I downloaded and installed the SMB server, and created a zfs filesystem for sharing stuff (which is covered in several places online, so I won't go into it here).

And the story comes to it's conclusion... I have a 2TB RAID storage server sitting on my network, keeping our photos safe and happily giving movies/media to TVersity (on my primary computer) to serve to my PS3.

Next steps:

• Before anyone mentions that my stuff still isn't 100% safe -- I'm in the process of figuring out off-site storage for the photos. I may write something (or find something someone else has written) to synchronize my photo filesystem with flickr.com.


• Obtain another 2.5" hard drive and add it to the operating systems zpool as a mirror (I'll probably have to unplug the optical for this).


• Get a low-cost managed switch and set my two gigabit network adaptors up with link aggregation.

[edited because I accidentally labeled this "Part 2" instead of "Part 3"; I also forgot a couple commands in the configuration part]

Building the *!#&@ thing (The NAS Project, part 2)

[covering assembly]

It was a bit like Christmas -- I got some UPS lovin' and had a great time unboxing all the components I received.


First and foremost let me describe the Chenbro case I ordered -- and I HIGHLY recommend you read the LogicSupply review on it.
They have some essential notes on building with it, as well as some cool pictures:
http://www.logicsupply.com/blog/2008/03/27/the-chenbro-es34069-case-review-part-1/

The Chenbro ES34069 is pretty cool -- it has a fairly small form factor (10"x10"x5" or so), fits a mini-itx mobo, a 2.5" system drive, 4x hot-swappable 3.5" drives, and a slimline optical drive.
The motherboard I went with, the KINO-690S1 supports 4xSATA 2, IDE, 2GB RAM, and a 64-bit AMD processor.

As noted in the LogicSupply blog, the fan that comes with the case was designed for the motherboard I originally wanted -- which won't fit my KINO mobo.
Luckily, I ordered the heatsink/fan combo from them that does fit.

I'll describe how *I* built this thing, and then note how it *should* have been built.

• Take everything out of the boxes.
• Take side panel off case.
• Take front off case.
• *NOTE: there are 5 little plastic tabs holding the front on. 3 of which you can reach by taking the side panel off, but to get the last two you need to pull the drive caddys out and reach your finger in. Hope you have small bendy fingers :)
• Test fit mobo in case
• Snap back connection template panel in case.
• Screw mobo in.
• Realize the heatsink needs to screw in to something. Oh, there it is... and it goes under the mobo. Sigh.

• Unscrew mobo.
• Put heatsink socket thingy in place.
• Screw mobo in.
• Assemble heatsink to CPU w/ silver paste.
• Screw cpu/heatsink/fan assembly to mobo.
• Attach 40-44 pin adaptor to 2.5" laptop drive.
• Realize there's only a single molex available to power both the 2.5" drive & the slimline optical.
• Do some creative re-wiring (molex Y-adaptor would have been nice, but I didnt' have one on hand at this point).

• Wonder where the system drive goes... (look up some pics online). Oh. Dang.
• Realize I can't get the 2.5" drive in place.
• Unscrew mobo (unnessesary if I had found the LogicSupply blog entry prior to this & realized I could pull the mobo tray out).
• Place system drive & screw in.
• Screw mobo in.
• Prep slimline optical drive - screw the adaptor into the back, screw assembly into removable drive tray.
• Realize the flat IDE cable I was using wouldn't reach with where I had the 2.5" drive pointing.
• Unscrew mobo.
• Remove 2.5" drive & rotate 180-degrees (so back of drive is under cd drive).
• Screw mobo in.
• Damn, forgot to screw drive in.
• Unscrew mobo.
• Screw 2.5" drive in
• Screw mobo back in.
• Shoehorn the IDE cable in place.
• Put RAM in.
• Hook power cable from case to mobo.
• Hook CPU fan power up.
• Move CPU fan power after noticing that CPU fan power connector was in a different spot.
• Hook up switch & LED wires from case to the header on the mobo.
• Realize that the Power LED header is 3 pin (live, center null, live), and the wire is 2 pin.
• Do some creative re-wiring using a jumper block I had on hand.
• Test fire for POST, etc.

IT BOOTED! WOOT!

After the fact:
I replaced the flat IDE cable with a round cable I ordered. I had to cut the little rubber boots off the round one to get it to fit, but it works much better than before.
Also as you'll read in part 3, I had to remove the hot-swap sata backplane... which would have been much easier if I had done so before installing anything in the case.

So assuming you'll be running Opensolaris like me, here is how I would recommend putting this together:

• Completely disassemble case:
• Remove the side panel (there's a thumb-screw in the back).
• Pull the four drive caddys.
• Remove the front panel (don't force it - remember there are 5 plastic tabs).
• Remove the mobo tray (unroute the cables).
• Remove the metal perforated backplane panel.
• Remove the two backplane boards from backplane panel -- they're not compatible with opensolaris. Bye bye hot-swap :(
• Mark & cut notches in panel for sata data/power plugs.

• Re-install modified backplane panel.

• Snap the back connection/port template panel (comes with the mobo) into the case.
• Screw the 3.5" drives into their caddys.
• Slot the 3.5" drives/caddys into the case.
• If you notched the backplane panel like I did, you'll want to:
• Cut the zip-ties that hold the cable bundle together.
• Replace the data cables that go to the top two drives with the ones that have right-angle connectors on one end.
• Plug the various sata/power cables in (you'll need the molex to dual sata power adaptors).
• Re-zip-tie cable bundle if desired.
• NOTE: Both the 2.5" drive location and the optical caddy have little metal pointy bits that are supposed to take the place of the screws (occupying the holes) on one side.
• NOTE: You will need to jumper one of the drives to be the slave. I couldn't find the right jumper on the optical, so I set the 2.5" drive to slave instead.
• Screw 2.5" drive in place (with its adaptor) -- remember, back of drive is located under optical tray area.
• Assemble the slimline optical drive: adaptor screws back of drive, drive screws into removable tray.
• Prep the round IDE cable (cut off the boots) & plug in to the system & optical drives.
• Build yourself a 3-pin plug for the power LED. I glued a jumper oriented 90-degrees to one side of the 2-pin led cable end, and ran a wire from the center to the side.
• Put the heatsink/fan socket thingy on the bottom of the mobo.
• Screw the mobo on to the mobo tray.
• Route the cables back through the mobo tray.
• Screw the 3.5" storage drives into their caddys (remove the black plastic placeholder)
• Slot the drive assemblies into their locations
• Plug the SATA power & data cables into the drives
  
• Screw the mobo tray back in to the case.
• Assemble the CPU/heatsink/fan & screw on to mobo.
• Add RAM to mobo.
• Connect the rest of your cables (good luck w/ the USB headers for the front of the case... I didn't end up getting them working correctly.).

You'll find some pictures I took during the assembly process out on flickr.com
http://www.flickr.com/photos/benjaminellison/sets/72157605600122163/

That should be it. Stay tuned for part 3, or "WTF, doesn't Solaris speak Linux?"
[edited to add some pics & a few assembly points I had forgotten]