Ok, so not quite so much voodoo yet... but I have other tweaks in mind for it, and they promise to be more fun.
Background: I have a new short domain name (flw.bz) resolving to the same place my original domain does (flowbuzz.com). I decided to keep my flw.bz files in a subfolder (/short). Oh, and my domain/webserver is virtual (and I think shared). I'm using YOURLS at the moment to provide URL-shortening capability (yay!).
Problem: How to get requests for flw.bz/blahblah to resolve properly while maintaining functionality of Yourls, not ending up in a mod_rewrite endless loop, etc.?
Answer: I tweaked the code in the .htaccess file that came with Yourls to look like this:
RewriteEngine On
# this doesn't do anything for me due to way my server is set up, so I left it commented out
#RewriteBase /
# take anything that's coming in to flw.bz
# and if its not a real file or directory, map it to short/ subfolder
RewriteCond %{HTTP_HOST} ^flw.bz$ [NC]
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-f
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-d
RewriteRule ^(.*)$ short/$1
# if I've put flw.bz/ in front of another URL, shorten it like a bookmarklet
RewriteCond %{HTTP_HOST} ^flw.bz$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} ^/(http://|https://)(.*)$
RewriteRule ^(.*)$ flwbz/admin/index.php?u=%1%2 [L]
# take anything that's coming in to flw.bz
# and now that it's mapped to short subfolder,
# if it *still* isn't a file or directory send it to YOURLS
RewriteCond %{HTTP_HOST} ^flw.bz$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^short/([0-9A-Za-z]+)/?$ short/yourls-go.php?id=$1 [L]
I may continue to do some tweaks, and probably will write my own PHP catching page and just use the Yourls API page. But after many hours of glazed-eye fiddling with complex mod_rewrite rules, this turned out to work.
[edited to add http catching code]