Those wanting to get a good overview of the microwave “e-band” landscape might want to check out this whitepaper.  Interestingly (from the whitepaper), “The 10 GHz of spectrum available represents by far the most ever allocated by the FCC at anyone time, representing 50-times the bandwidth of the entire cellular spectrum.”

Near the end of the article is a little blip about technical and regulatory characteristics.

A bit dated (e.g. some wireless vendors handle mobility quite well now), but a good intro for those who want a quick intro to the whole wireless scene.

(1) Ensuring people are who they say they are, via government issued IDs (aka the “assurance” process).
(2) Giving them “assurance points”.
(3) Giving them an online tool which they can now use to create PKI certificates for certain lengths of time, given the amount of “assurance points” they’ve racked up.

The biggest downside to CA Cert is that the cert is not officially recognized by browsers, so getting it to work means going to their web page and knowing to download and install the root cert (e.g. this PEM file). Then after you do that, you must login the website, confirm all your email addresses, generate private keys for those email addresses, and then know how to import all of those keys into your various applications. Unfortunately, many people don’t instinctively know how to do all of these little steps, so it’s not very accessible to someone who wants something quick and easy.  As with most community driven projects, it’s a work in progress that is a labor of love for those who like it, and a PiTA for others who want something quick ‘n’ easy.

It will be interesting to see where this goes, and I’d be curious what others think about the project, as well.

Most of the time, there is nothing you can do to wait it out once you’re delayed, but in the cases where you have a bit more time and might plan an alternate route, one of these websites might prove helpful.

• Flight Delay Information
• Real Time Flight Tracking
• More real time flight tracking

• a cheeseburger in a can
• bacon donuts
• this weirdass worm
• bad, bad horror movies (e.g. The Abominable Dr. Phibes
• mixing Catholicism with indigenous religions
• Hitler remixes (Notorious H.I.T., Hitler Rap)

So, a coworker called me the other day and said (something like), “What are you doing at 2:30pm today?”  l told him I was open, so he invited me to an online webinar for Nu Skin.  Having recently been put in the position of having to giving presentations (something I’m not naturally inclined to do), I decided to check it out and could not stop laughing throughout the whole presentation. The presenter had this douche-y MLM “scam” voice where he talked fast, rattles off half baked statements, fired off numbers, and then periodically asks rhetorical questions designed only to provoke the one and only response he needs to set up for the next slide.

It’s really hard for me to understand why so many people fall for this bullshit. Intellectually, I suppose, I can only compare it to my lust for poker.  Poker is ruthlessly predatory, and the difference between poker players and your “uplink” in a MLM scheme and a poker opponent is that poker types are more honest about the nature for the game (to leave the table with YOUR money).  I find MLM schemes pretend to be something better, but then when you look at how exactly money is paid out, you’ll find it very leveraged to the upper tier, blurring the lines on what is a pyramid and what is a legit business. (Telling people that they’re not a pyramid scheme always sounded a bit disinginuous to me, sorta like telling your kid brother, “I did not punch you; you ran into my fist!”)

One of the best pieces of financial wisdom that someone imparted on me was, “He who strives to get rich in a year will get hanged in a month.” The MLM folks who are successful seem to often do so at the expense of whatever political capital and cred that they had with their friends.  At least with poker, you know what you’re getting into. After all, the famous Amarillo Slim once said, “It’s immoral to let a fool keep his money.”  MLM people think like poker players, but are just too “nice” (or savvy!) to put it that crassly, I suppose.

Perhaps the best poker adage that applies to MLM bullshit is, “Look around the table, and if you can’t tell who the sucker is, it’s you.” If you’re “uplink” is telling you it’s a good idea, and you don’t have “downlinks” to fuck over, then maybe you should stick to something that requires a few fewer brain cells!

From what I can tell, the bulk of the plugins are dictionaries and spell checkers, but while sifting through them, I found a gem called MultiSave.

After you install it, you get a little multiple disk icon thingee in your menu bar.

After you configure your settings, you get an extra option in your toolbar to ’save all’ (unfortunately, no keyboard shortcut).

Pretty cool!

Needing to collaborate with several other network guys who didn’t have a cool screen shot tool, I did a little searching to see what I could find and was pleased to find ZScreen and Greenshot. Both are small programs. Zscreen is a bit more full featured than Greenshot, but Greenshot is less than 150K which fits on older hardware. Zscreen, for better or worse, integrates with imageshack.com, which is kind of nice in certain situations. (Screenshots of Zscreen HERE)

For these types of things, three tools come to mind.

• SpaceExplorerEx: finds space hogs on a hard drive
• WinDirStat: also finds space hogs (but with a more colorful GUI)
• DriverBackup! 2: backs up drivers for later

I’m kicking around the idea of creating a wiki where people could post FOSS tools that they’ve used for servers. Any ideas others have on the subject would be welcome.

In asking how it will be used, I ask questions like,

–Is this outdoor, indoor, or both?
–Is there roaming?
–How much bandwidth is needed from client-to-AP and from AP-to-AP?
–Where is the egress points for the WAN?
–How is this being paid for? (This gives me a very good idea how serious they are in building something well)

When asking these questions, I eventually get a sense of what they’re trying to do with wireless, and then I have a fairly good idea of the best products to use for their wireless solution. Ideally, I’ll also have a sense of what the limiting factors are. Sometimes, it’s not so obvious, and it’s helpful to ask what others have said (particularly other vendors) was the main problem with the deployment. Chances are that it’s some variation of something like terrain, environmentals, installers, budget, power, mounting rights, performance, coverage, aesthetics, etc.

I find that if I do not nail them down on the specifics of whatever is the most limiting factor, then I run the risk of having to do lots and lots of extra work once I find out that I have less options. For example, in a recent survey, I was told I had the poles to work with. I planned out a channel plan and brackets that would be most appropriate for that scenario, but then towards the end of the survey, I was told that maybe the poles were not available, due to everything being on the same circuit and also some concerns about aesthetics.

The poles are perfect for the area. On both sides of the poles are 6 story buildings, and 6 dbi antennas give me over almost 50-100m radius in some of top most rooms. Take out that mounting option, and I risk having to plan out building mounts which mean, (a) they’re harder to implement on a 5 story building, and (b) they’ll require more (building being covered will likely have too much Wi-Fi, leaving other areas that would otherwise be covered with an omni antenna too cold).

Budget is another common example. People often want great coverage but aren’t willing to pay for it, so I’m left with only putting single radios in areas with egress points or (ugh) meshing on single radios. And sometimes I might find the absolute perfect spot to mount, say, a central radio that I can PtMP with, however I soon find out that is unavailable due to the limitations or shortcomings of the installer.

For example, consider a campus environment where you have tons of student devices slamming access points, such as large conference areas. In these environments, you have lots of collision on the 1 through 11 on 2.4 GHz 802.11b/g (and soon, 802.11n) channels. In these situations, it’s better to split out those users on different access points so that they don’t (a) congest the 2.4 GHz spectrum (1, 6, or 11 in most indoor cases; but often 1, 4, 8, or 11 in large outdoor muni mesh environments), or (b) crank down the power in certain areas to force the users to connect to the “correct” access points.

Also, why blast out the signal and risk causing the laptop to get a signal, but not be able to send *back* the signal? I see this all the time, and it’s frustrating to users. I tell the wireless admins that Wi-Fi devices listen before they talk (CSMA/CA), so if there is noise out there from another BSS stations on that channel, that client device will wait until there is an opening. And when there is an 11b client in the area, everyone on that WAN suffers (more info here).

So, you’ve got two choices here: use a good radio that can receive faint signals of distance laptops that receive your signal or stop cranking up the power on shitty radios. Seriously, folks…less is the new more here.

Install the plugin, and then next time you log in the admin part of Wordpress, you will be alerted if there is a WP update. Click through the little wizard and it (literally) automagically downloads the newest software, saves a backup of your databases, puts the site in maintenance mode, and then checks to make sure everything works okay before making it go live again.

Every day, I get bombarded with new .ru users, and Akismet picks up thousands of spam comments. I’m quite sure that if I didn’t keep up with the security updates, one of these ruskies would have most certainly done something dastardly.

In short, Google Earth sucks for integration due to the way that they change URLs, and that means that you might create an app that works today and breaks tomorrow. OpenStreetMap is almost as good (if not better in many ways) and completely free. Its tile-based design lets you zoom in and examine streets in more detail, and you can contribute your maps to the project.

Those interested in knowing more might check out the following resources:

• Beginners Guide
• FAQ
• Map Making Overview

Here are some interesting trends I’m seeing:

Tell people their resumes suck, and they will still try to convince you that it’s “good enough”

As I told one of my best friends recently, imagine playing musical chairs, and all of a sudden half of the chairs go away in one fell swoop. You simply cannot assume that whatever strategies and tactics that worked before will work now.

If you’re feeling lucky, rock on and tell me where I can shove my advice.

If not, then reconsider taking some advice and redoing your resume before your unemployment money runs dry.

Few people ever see the value of the sniper approach

If a job is worth you applying for, then it is worth an hour or two (or more!) totally tailoring your resume for.  Period.  No, seriously…I don’t want to hear your lameass excuse, so STFU.

For the job your applying for, strip out whatever skills that you think are liabilities or might detract from your ability to do that specific job (i.e. Linux skills in a Microsoft environment, and vise versa).

So, if the job is with a manufacturer, then maybe focus less on the budgetary and day-to-day stuff and more on technologies, skills, or contacts you have that directly relate to that manufacturer.  If you’re applying for some sort of in-house systems or network admin position, then I will instead focus on being reliable, special projects, making budgets / deadlines, etc.

Your resume (particularly the ’summary’ section) is not the place to brag about IRC channels you frequent (yes, I recently saw that one) or the fact that you’re tewtally a total social animal who frequents every single geek luncheon within a 100 mile radius (lots of resumes have this stupid shit. Why?).  Does this mean you’re cool? Yeah, probably.

What you say in your resume tells an HR person what you consider is most important about yourself, and when you list events you go to, it conveys the idea that you’re good at going to events.  Are you a community organizer or something?  No?  Then leave that shit out!

People who finish college tend to summarize their experiences better, even though they lack experience

When I tell people to think of their summary portion of their resume as their “thesis” and to make the rest of their resume support that central idea, college graduates tend to do extremely well.

The irony is that while their resumes tend to be “good” (i.e. no typos, tighter, more relevant to the job, etc), they tend to be less qualified candidates, IMO.  The non-college graduates tend to have more tangible skills, but since they cannot seem to articulate them worth crap, I’m betting that they’re likely not going to get noticed unless they have something else working in their favor (contacts, a practical test, good interview, , a good headhunter, etc).

People without a college degree tend to focus too much on details

Perhaps this is not a fair generalization, but I notice that people who did not finish college / university tend to just regurgitate all the shit that they can do in some weird order that makes little to no sense.  Knowing some of these people personally and what they might contribute to an organization, I would definitely hire them over the ones who did not attend formal training.  But if I were an HR person, it would be hard for me to really see their true worth amidst big blocks of acronyms.

Seriously, if you’re going to detail all of your skills, then consider framing or organizing them in a way that makes sense.  A shitload of bullet points you arbitrarily put down is just really annoying.

Playing it safe on resumes makes you look dull

Stop saying that you’re “goal-oriented”, “hard working,” (insert some stupidass, overused phrase HERE).  Seriously, just stop. If you are a badass at something, say it (hell, even use the word ‘badass’).  Take a risk and let your personality come through in a resume.

People look for skills and passion. In the IT business, I look for someone who truly loves challenges and is always looking for new problems to solve.  I look to work with people who are cool to work with in fucked up situations.  Show the employer that you’re that type of person, not that you can simply occupy a desk.

Not focusing on your last two or three jobs is a big liability

If someone is going to evaluate your capacity in a new job, your last job or two is perhaps the best indicator of your performance in your new role.  Given that, why do you insist on giving equal time to, say, to some burger flipping jobs you had years ago?  Were you manager or is there some other compelling reason you’re including something that’s totally unrelated to the job you’re wanting here?

Your work history paints a picture

So, you had some restaurant job for several jobs and then you just leaped into a network engineer job?  C’mon…

Seriously, what is an HR person supposed to think when they see that?  Were you in school during those restaurant jobs?  Or is there some other compelling reason to not think that this last job is, oh I dunno, a big stretch for you?

Resumes that focus solely on responsibilities suck.

Stop listing all the stuff you “troubleshooted, configured, and maintained.”  Instead, list what you accomplished and what the bottom line benefit of the company was.  Anybody can just sit in a room and dick with solutions, but did your dicking around result in the benefit of of the company?  If so, great.  If not, then omit that and focus on those bigger accomplishments.

So, SRSLY…why WiMax? Much of WiMax’s “unique” offering could also be done with Wi-Fi. Unless you are a carrier who wants to use that band, lest you lose your exclusive rights to it, or perhaps your backhaul bands are totally slammed, why?

My conversations with “engineers” at big carriers and integrators go something like the following…

RF engineer: “So, do you guys do WiMax?”
Me: “Yeah, we have a radio module for it. But what are you trying to do, and do you have a license?”
RF engineer: “No, but we’re trying to get get lots of bandwidth at a long range and think that it might be a good fit.”
Me: “Well, how far are you trying to go and what bandwidth do you want at that distance? This can most likely be done with Wi-Fi. Besides, you can’t really have distance *and* bandwidth at the same time with RF.”
RF engineer: “Well, how far can Wi-Fi go?”
Me: “We gotta nail down some specifics of a scenario, and we can calculate it now together if you give me some details, then we can compare that to what WiMax should do in the same situation. In fact, give me a situation and I’ll tell you what sorts of antennas and bands we could use.”
RF engineer: “Well, we just really want WiMax.”
Me: “Uh, okay…You will have the same crappiness with unlicensed WiMax that you do with unlicensed Wi-Fi, so let me know how that works out for you. It’s 2.3/2.5 GHz which is really close to Wi-Fi’s 2.4 GHz RF characteristics.”
RF engineer: “Alvarion has WiMax, we hear.”
Me: “Great, let me know how that works for you. Oh yeah, and I hope you enjoy paying like $700 for a CPE. It’s not like you can just go to Ruckus and get one of those puppies.”

I have had variants of this conversation over and over and over again. It’s like no one knows why they should have WiMax, just that they know it’s the new new thing and that it should magically help solve all of their problems. In defense of WiMax, it has some cool QoS features (or so I hear), but I have yet to hear people mention these, and I cannot help but laugh when they insist that WiMax is the only way to go when they are sorely lacking in any knowledge whatsoever on RF propagation.

Perhaps most amusing here is that the city technologists that seem to be the big proponents of WiMax adoption. It’s like they are itching to cut their teeth on a new technology or are itching to make a name for themselves on some spanking new wireless rollout. Operations people who have (a) use their budget to pay for the infrastructure equipment, (b) implement and maintain it, and (c) rely on it for other network services tend to have a more realistic view of WiMax’s role in the universe, IMO.

For all you WiMax people out there….

They refused to go to training on the products they were going to implement, and I wish that we had ultimately just dumped them, but it wasn’t my call. The customer was not familiar with outdoor wifi (even though they had “wifi” in their name!), nor where they familiar with carefully troubleshooting network stuff one layer at a time.

Usually, I’m pretty cool about helping people who have no clue, but in this case, el presidente duderino had a major attitude and liked to chew on people who tried to help him but who weren’t officially tasked with doing so, which is a sure fire recipe for starting out a job on the wrong footing.

My conversations with them would go something like this.  I wish I was only joking.

layer 1

Me: “Can the antennas “see” each other?”
Him: “Uh, I think so.”
Me: “Great, show me.”
Him: “What?”
Me: “Show me the following: your Google maps, your link budget, and the pictures you took while standing on the roof.”
Him: “Oh, I haven’t done any of that. You guys did that.”
Me: “No, we made pre-design showing you what it looked like from Google earth and attached an approximate cost for that hardware. That was almost a year ago.”
Him: “So, you’re not going to do that all of that this week when you come out?”
Me: “No, you need to do that. I only configure the equipment.”
Him: “So, can we do that after you configure the equipment first and then we’ll do it?”
Me: “Uh, no…the way I configure the equipment depends on what stuff can see each other. Remind me again of the value do you bring as a VAR?”

layer 2

Him: “The mesh isn’t working.”
Me: “Be more specific.”
Him: “I can’t see the other radio.”
Me: “What do you mean by see?”
Him: “I can’t ping it.”
Me: “Ping is ICMP, and that’s layer three.  Have you set up the channels and stuff?”
Him: “I dunno, I guess.”
Me: “Our stuff doesn’t automagically configure itself. The radios you pointed towards each other have to be told what configuration they’re in — PtP, PtMP, or MPtMP and have consistence channels, consistent link topologies, same encryption settings.”

(later)

Him: “Ok, I did all of that and I cannot ping the radios.”
Me: “What is your IP address?”
Him: “I dunno, 192 something.”
Me: “What is the IP addresses of the radios?”
Him: “10 something.”
Me: “And you have multiple IPs on your NIC? Or do you have routes?”
Him: “I didn’t set up the router.”
Me: “Then you’re probably on the same broadcast domain, so either (a) change all the radio IP addresses, (b) put muliple IP addresses on your NIC card, or (c) do something upstream to make sure you can route to this other subnet.”

(much, much later)

Him: “Ok, I did all of that.  I want to do VLANs and RADIUS and make sure all the users can never see each other’s traffic.”
Me: “You’re telling me that you (a) have not done speed tests, (b) have not set up all the mesh nodes yet, and (c) have not put in any sort of real monitoring solution, and you want to start tightening the screws?”

layer 3

Him: “I still cannot ping stuff.”
Me: “What do you mean by stuff?”
Him: “I fix the channel stuff and then my pings stop working.”
Me: “You probably broke something layer 2 related and now cannot pass your ICMP traffic.”
Him: “Oh, how do I troubleshoot that?”
Me: “fping your entire network before you start screwing with stuff, then toggle back to that screen and see which hosts died as a result of your ‘fixing’.”

(later)

Him: “So, everything is up, but certain areas cannot see each other.”
Me: “You just couldn’t resist playing with VLANs, could you? Turn that off. We’ve got to get all the layer 2 areas up before we start thinking about routing traffic between then.”

layer 4

Him: “Users can surf the Internet, but I cannot ssh into the radios from home.”
Me: “You have a firewall?”
Him: “Yeah, I think so.”
Me: “Well, call me when you know so, and we’ll talk about your ingress and egress policies.”

layer 5+

Him: “So, users are killing my bandwidth with crazy stupid traffic”
Me: “Our units are only layer 2, so you can only QoS with 802.11e.”
Him: “Ok, so configure that to stop torrents.”
Me: “It doesn’t work that way. Go buy a Packeteer and call me when you’re done racking it.

(later)

Him: “Packeteers are expensive.”
Me: “Then hire me for lots of money and I will build you a ‘free’ solution.”

About 2/3 of the time, a simple 3 paragraph email suffices: one sentence telling them directly what I need, three sentences describing it in more detail, and a final one sentence paragraph describing the follow up.

(Verbose emails by cow-orkers automagically go to the bottom of the day’s queue)

“It’s like an Olympic athlete…who’s been struck down with necrotizing fasciitis, the “flesh-eating” disease. (And imagine the government as leg weights.) Cure the credit markets and he can get back in the game.”

Last week about a dozen or so people I know got canned after the stock market took a dive, particularly those in small businesses that depended quite heavily on credit to make payroll. There is no end in sight, and it’s going to get worse before it can get better.  There are still lots of kinks that need to be ironed out, and today’s interest rate drop will likely do very little.

While I cannot do much about the overall economy (other than vote Obama), I can perhaps help others who are in the IT space and are in desperate need of a job. I know some good people in the industry — headhunters, telcos, VARs, and VCs — who are always looking for good talent.  I can’t guarantee anything, but I am happy to help make connections if the match is right. Just send me a email with your resume, along with some contact info and the type of job that you are looking for.

Before you send me anything, however, just know in advance that I will likely do the following:

–ask that you improve various things on your resume before I forward it on.  These things might include: telling me exactly what you’re looking for, focusing your resume, making your resume more results oriented, clearly bulleting your skillset, etc.
–ask that you do the same for others who you know are in the same situation.
–ask that you take my time seriously.

If you’re cool with that, shoot me over what you have (roger AT hack my idea DOT com), and I’ll see what I can do on your behalf.

From a technical point of view, Google indexes your site incredibly fast. Seriously, like 2 hours after my post, it was indexed and ready for their Google alerts blast.

Also, Google’s Webmastering tools rock. Enter in your domain, put a file on your website to verify that you do indeed own that domain, and then you can do all sorts of cool stuff, like ask that URLs not be cached/indexed, analyze and generate robots.txt files, etc. I am not a web programmer, so most of it is new to me, but I find it cool how they have made otherwise difficult things easy for the lay person.

More importantly, from a business point of view, I learned the following:

• Some customers are anal about not letting you mention that they use your product.
• If the customer asks you not to talk, sales people can do so as much as you want, as long as the customer does not catch wind of them doing it.
• Sales people are good about creating plausible deniability, just in case they ever get caught talking.
• Sales people are better than technical people at not getting in trouble for talking.
• If something is not yet common knowledge, slip out just enough info to make it common knowledge, then reference other people that talk about it.
• When you cannot officially release a press release on something, link to someone else’s URL that says basically the same thing.

Now, please move along.  There is nothing to see here…

• Fresnel Zone Calculator
• RF Link Budget Calculator

People don’t just don’t realize just how far wi-fi can go (provided that spectrum isn’t already cluttered, which it often is).

For example, put a 23 dbi panel antenna on one of our both ends of our BelAir radios using one of the 11 outdoor channels available on 802.11a, and assuming a 1 dB cable loss on each end with 23 dB output power, and you get the following modulation rates with a 12 dB fade margin as your safety net.

54 Mps (-71.4 dbm) - 5.4 miles
48 Mps (-73.4 dbm) - 6.8 miles
36 Mps (-78.4 dbm) - 12.0 miles
24 Mps (-81.4 dbm) - 17.0 miles
18 Mps (-84.4 dbm) - 24.0 miles
12 Mps (-86.4 dbm) - 30.2 miles
9 Mps (-88.4 dbm) - 38.1 miles
6 Mps (-88.9 dbm) - 40.3 miles

If you want to be sure on areas with low fade margin, just put two of our radios on both ends of the long shot, and set the antennas with opposite polarization. Plug the radios into a good layer 3 switch, and the RSTP algorithm on our radios will take care of everything should one of the links go bad.

osTicket, while not nearly as extensible as something like, say, RT (Request Tracker), is a fairly straight forward ticking software program that does the “basics” well enough — autocreate tickets sent to email address (e.g. support@yourDomain.com), let you easily manage as assign tickets, pop the account that’s receiving the email, etc.

Project Pier is a sort of free PHP-based version of Basecamp, which is actually quite useful in team environments that need to work closely together and share the project data with clients. Project Pier is free (GPL), easy-to-install, and has almost all of the same important features as the pay version of Basecamp. I’m using it now instead of the other Basecamp-clone I switched to last year, activeCollab (HOWTO on upgrading from activeCollab to Project Pier (which I have not tried yet).

Akismet has caught 3,423 spam for you since you first installed it.

If you use Wordpress, you mos def have to check Akismet out. (They offer a free API key for those using it for personal use)

So, tonight I tested out Microsoft’s Office Live Workplaces, and while it doesn’t have the cross platform features that I’d really like to see (and have come to depend on with Google and Zoho), it does a somewhat seamless integrate with MS Word.

To sign up, go to the web site, sign in with your email address, login and install their little program thingee to integrate with Word, and voila! Now when I open up MS Word (at least with Windows), there is an extra button there that allows me to save to my online MS account.

(For giggles, I also installed MS Shared View, but haven’t yet used with anyone else, as everyone else seems to use tools like CrossLoop.)

Interestingly, once I dropped down below 10 mW, I could not get a DHCP lease from a laptop, even if I were a few inches away from the router.

(According to the dd-wrt FAQ’s “wireless questions” part, 84 mW is around the limit before you start to distort the packets [I have yet to see proof of this, though]).

In fact, tonight one of my associates, Dino, and I used it to co-write some proposals we’re presenting to some datacenters and integrators that we’re working with, as well as sketch out some ideas for some more Nagios articles on Techtarget.

In overall features offered, it’s quite cool. Just open it up (using your Google login, if you’d like) and you have all, if not most, of the formatting features you’d have in your word processing program.  The main reason I switched was because I needed a table of contents for our proposal, and I did not want to shotgun Word docs to all the people in the party after each edit. As long as you use the right headings (like in Word) the ToC autogenerates, as expected.

When an invited user logs in your document, you see their icon in the left, and when you click that person’s icon, you see a color associated.  That color (yellow in my case) then colors the paragraph that is locked for editing by that other person. The lack of this feature in Google docs can sometime be annoying, and sometimes you need to IM the person and tell them to stay away from your section to ensure that there is no problems.

Things were going quite well, until I decided to make an outline using bullets. This is where Zoho totally blows. For example, I often like to jot down ideas in a bullet format and then drill down into some of the ideas to flesh them out. Instead of simply letting me hit RETURN and then TAB, I had to use hit RETURN and then use the mouse to click the appropriate arrow to TAB in or out. I personally found this insanely annoying while trying to brainstorm.

Another big Zoho annoyance is the way that the ToC blows away any heading you might have at the top of your doc. I wanted to put a title, do a hard return (which I couldn’t find in the menu), and then have the ToC on the 2nd page. After about 5 min of looking, I gave up and resigned to just let Zoho just blow away whatever I had at the top of the page every time I clicked the ToC button to autogenerate based off the headings 1, 2, and 3 I was using.

Until I get all the wrinkles worked out, I suspect that I’m going to have to use a combination of all of the tools to do exactly what I want — Google apps for brainstorming, Zoho for collaboration, and Word for the final formatting touches.

(Supposedly, MS has some cool collaborative tools, but I have yet to use them.  Please comment if you have.)

To test it out, SSH to the remote host, making sure agent forwarding is on. Then try:

ssh-xfer somefilehere.txt

The file should magically appear at your local ~/Desktop, or on your Windows desktop with the PuTTY patch. If you want a different path, edit the patch. You can also go

cat somefilehere.txt | ssh-xfer nametocallfile.txt

Let’s now assume we want to possibly build out the clusters, which might increase the backhaul requirements. In this case, we might need to change from backhauling via 4.9 GHz (50 available MHz, 4940-4950 MHz) and using one of less crowded channels on the U-NII bands (lower: 5150-5250 MHz; middle: 5250-5350/5470-5725; upper: 5725-5825). For the sake of argument, we’ll pick the U-NII band, which as you probably know, is the same frequency currently used by 11a (and with a firmware upgrade in the near future, 11n, which will give us 3 non-overlapping channels on 2.4 GHz and 8 non-overlapping channels on 5 GHz, and a typical data rate of 74Mbps thanks to MIMO).

Before I propose a solution, I need to make sure a few things: (a) that wifi can make that distance, (b) that I can make that distance using appropriate EIRP (”power”) levels, and (c) I can push the right amount of bandwidth through that long distance (and if not, how many intermediary links do I need). Roughly speaking, here are some steps to consider:

1. Get the best directional antenna at your disposal, in my case, it’s a 23 dbi panel antenna. (These types of antennas often shoot their signal in a polarized form, and you can use simultaneous channels by simply turning the panel 90 degrees)

2. Find how much power you’re able to use legally with that antenna. There are two situations — PtP (point to point) and PtMP (point to multi point), each of which has differing requirements for 2.4 and 5 GHz. Skipping over the complexities and nuances of this regulation, we’re limited to 52 dbm (~160W) on the upper 5725-5825 MHz U-NII band) for a PtP link.

29 dbm intentional radiator power (in radio) + 23 dbi panel antenna gain - 2 dB for cable loss = 49 dbm, just a tad bit under the 52 dbm FCC limit

(Note: if you do not know the difference between dB, dbm, and dbi, check out this resource)

3. Next, we calculate free space optic path loss in upper U-NII 5.725-5.825 GHz part of the 802.11a band. This is the amount of dbi we lose due to the laws of physics (i.e. wifi being wifi, nothing to do with anything obstructing its path). It is dependent on distance (in mi) and bandwidth (in Mhz)

20 * Log10 (5725 MHz) + 20 * Log10 (5 mi) + 36.6 = 125.7 dbi

(Here is an online calculator, and here is how that formula is derived)

4. Assuming your Fresnel Zone is cool (i.e. nothing blocking the greater area between you and the link), you now need to calculate RSSI signal strength that reaches the other point that is 5 mi away.

The general forumula for this is…

tx power - cable loss + antenna gain - free space path loss + rx antenna gain =rssi.

assuming a 2 dB loss in cable resistance, we can now plug and chug…

29 dbm - 2 dB + 23 dbi flat panel - 125.7 dbi + 23 dbi flat panel on other end = -52.7 dbm

5. I know look at the sensitivity rating of radio I’m going to use. In this case, our BelAir radios give the following mudulation rates at various RSSI levels.

54 Mbps: -71.4 dbm
48 Mbps: -73.4 dbm
36 Mbps: -78.4 dbm
24 Mbps: -81.4 dbm
18 Mbps: -84.4 dbm
12 Mbps: -86.4 dbm
9 Mbps: -88.4 dbm
6 Mbps: -88.9 dbm

6. Look at percentage lost because of distance. The general rule of thumb here is the 6 dB rule (3 dB is 2x, so -6 dB is -4x), where every doubling of distance results in a loss of 6 dB (this breaks apart a little when you compare 2.4 GHz vs 5 GHz, but it’s still more true than not true). I have a little cheat sheet, which shows me percentage I lose in terms of km.

I convert for miles (5 mi * 1.6 km/mi = 8 km) and then look on my chart to find the 10km mark. At that distance, I only have 52% of the 54 Mbps modulation rate, from step 5. “Good put” (what the radio actually puts through) is about 24 Mbps, so 52% of my goodput is, approximately, 12 Mbps.

7. Steps 1 through 6 assume we have an 11a radio, which is only sort of true. The BelAir ERM radio uses the atheros chipset, which gives us a few 11n-ish features, such as packet concatenation. The “good put” on these radios is around 35 Mbps, so dividing that in half leaves us a little wiggle room for the anticipated 16 Mbps (4 x 4 Mbps) that each cluster will have to backhaul 5 mi away using our 23 dbi antenna.

1. Install the External Editor Version xpi plug-in (I used 0.7.4 on Thunderbird 2.0.0.14)

2. Install VIM (or whatever external editor you prefer)

3. Open up the add-on options for Thunderbird and select, say, “C:\Program Files\Vim\vim71\gvim.exe” (and any other mail options you want to integrate with Thunderbird).

4. Fire up a new email and hit control-E to open up your external editor.  When you’re done, close the editor (e.g. :wq for VIM users), then control-ENTER like you always do to fire off the email.  Anything you put in that editor is sucked back up into your Thunderbird email compose Window!

fping 192.168.1.100 192.168.1.101 192.168.1.102 -c -D -L c:\path\to\file.txt

With this command, you can ping devices continuously with a time stamp, and when you’re done, you can simply control-c to end the parallel pings and have the file.txt as material for later review.

(The results may not be as customer facing as you might like, but when you need to quickly test something for you own sanity, the command is quite useful!)

For example, I just spec’d a ~$70K (retail) dual radio solution BelAir for a condo in FL. If we turn this opportunity to a MSO, then they get, say, everything for ~$50K with whatever monster MSO discounts MSOs typically get. There are about 100 condo units, so roughly….

$20/unit * 100 units * 12 mo = $24K/year the MSO earns.

So, in about two years, the MSO (in theory) recoups the cost of their CAPEX.

In cheaper condo environments (<$300K), particularly any sort of condo with only seasonal visitors, an open-mesh or Meraki sort of solution might make sense, as that way owners do not have to worry about monthly commitments. These units paying, say, $5/week or $20/mo on a per transaction basis pay off the infrastructure in a matter of months, with almost zero OPEX  (but with the added cost of having Meraki take their 20% for their automagical billing solution).

Open-mesh looks very promising, IMO. While I haven’t seen any billing solutions that integrate with it yet, I hear that several 3rd party solutions are on the near horizon. Rumor has it that both Meraki and open-mesh both have insanely easy-to-use monitoring solutions for their product. If that’s the case, I would love to see one in action, as this is something that is insanely expensive with other vendors.

On a slightly different note, I’m getting good reports from various associates and colleagues about various CPE gear, particularly the Mediaflex and Metroflex. Someone I know just mass rolled these out in an apartment in the Phoenix area, and the tenants he says seem quite happy.  As far as Meraki goes in this department,  a quick google search on “meraki openwrt” and “meraki netbsd” shows some interesting projects being kicked around (which I saved on my del.icio.us profile, for those interested)

After you install the plug-in, simply right click the folder with the dupes, and select “remove duplicate messages”. Le voila!

Any pointers in the right direction would be greatly appreciated!

On the third day (over 24 hours after the incident), I made a last ditch effort of just disabling the IMAP from within Gmail.  Sure enough, that fixed it about an hour or so later when I checked it.

I got a similar Gmail lockout a few months ago, that time for what I suspected was my Greasemonkey tomfoolery.  In that case, however, I was back in my account in a matter of hours, not days.  :/

Outside each hotel room, I would easily get ~12 Mbps, yet as soon as I came in the room (right by the window), my speed might drop to 9 Mbps, and on the first bed, I might get only around 6 Mbps on the first bed. The second bed might get 4 or 5 Mbps, and the sink area would drop to less than 1 Mbps. Bathrooms were the worst — around 300 Kbps or so. Interestingly, I almost never lost a continuous ping the whole time.

Today we easily light up the faces of several hotels with just one cheapo BA100 unit with the default internal antenna with a 9 degree tilt. Our second hotel today was a textbook case. One unit on each side of the 9 x 14 room area gave us solid coverage, and a unit on top with LOS to each unit provided the egress and backhaul to the downstairs units. As an added bonus, we found a perfect little water resistant cranny for the unit to permanently live on top of the roof!

In theory, it should have all been very simple.  All I did was upgraded from v23 to v24, but for whatever reason shortly after the upgrade, none of my wireless devices could use DNS properly. I downgraded back down to v23 and everything seemed to be working fine for a while, but then about 2 hours later I got locked out of my router (for no particular reason).  I did a hard reset, upgraded to v24, and everything has worked quite smoothly since.

One of the main reasons I upgraded was to support virtual SSIDs, so now I can have multiple SSIDs, one broadcasted SSID with WPA for me, and another one not broadcasted with WEP for guests.  I believe one can choose *not* to bridge that other SSID to the other networks current network, but I have not tried that feature yet.

One of the cooler features of dd-wrt is QoS, and for whatever reason, QoS on v24 seems to be a lot more functional than previous versions.  Using v24’s QoS, I labeled all torrent traffic as BULK, then I classified DNS, IMAP, SMTP, POP, and HTTP traffic as EXPRESS. For good measure, I labeled ssh traffic and my iPhone’s MAC address as EXEMPT, so now checking email during torrent-ing takes only a fraction as long.

As a proof of concept, I’m torrenting like hell right now, and my somafm.com stream hasn’t even missed a beat — something unheard of until today!

Insidder helps identify sketchy coverage (e.g. areas below -80 dbm RSSI). Its real time RSSI graph of the various SSIDs helps wrangle the 802.11 channel craziness by finding which channels are being tied up on neighboring SSIDs. Sometimes, if I have adequate coverage in an area and a laptop often jumps on a neighbor’s stronger SSID too soon, I might sometimes drill down into their driver and tweak the roaming aggressiveness. And for when you never want to roam to a neighboring AP using your SSID, I have begun started hardcoding MAC addresses to the actual SSID, which you can do in Linux as well as Intel PROset/Wireless program for Windows (haven’t done it in OS X yet).

I was talking with one of the other SEs in my company about the current state of wireless resembles the early days of ethernet (at least, based on what others have told me). Everything is a sort of voodoo, and finding good, simple to read resources / tutorials is not yet easy. In order to figure out the underlying protocol, you often have to sort through gobbledygook or hype, rather than going to a few key easy-to-read sources. As wifi gains momentum (particularly over wimax), this will change, I think. But until then, actually understanding the alphabet soup that makes a wireless protocol and its relation to applications (e.g. how collision avoidance affects VoIP) might be a little murky.

(1) Give all taco trucks a GPS unit, aggregate all the live data on some web backend, and then display it in some sort of a google maps mashup that’s phone-friendly (e.g. SMS your address, and it SMS’s you back driving directions from your address). Show me a single late night hispanohablante clubber who wouldn’t be all over esta mierda!

(NPR had a special a while back on a website that mapped out various taco eateries, but I’m really like to see something more real time that’s interactive with people with cell phones who want late night eating options.)

(2) Take all of the major datacenters in the United States (e.g. in SoCal, you’d take Hosting.com, One Wilshire, Savvis, etc) and then make, say, a wiki entry around each one that tells you about local eating and parking options. So, at One Wilshire, you might put The Pantry and then note that it costs $1 to park and that you may have to compete with people coming from The Derby. You might also note caffeine and parking options in the area, as well as primo parking spots for when you (a) don’t want to spend very much money, or (b) need to unload a lot of gear but don’t want to pay the $30 or whatever to park in the underground areas.

(Perhaps one of the first places you start with is some of the major carrier hotels that companies like CRG West own)

In one sense, I can understand how IPtables is a godsend, particularly in environments where network admins are slow to make necessary firewall changes or are (understandably) reticent about giving others access to networking equipment. On the other hand, ad hoc configuration kludges everywhere can get insanely unwieldy, and should the sysadmin leave unexpectedly (which I see all the time, particularly in high pressure data center environments), the next sysadmin who takes his place (and more importantly, the company!) is stuck with some major firewall craziness to sort out.

To deal address both of these problems, here are some solutions I have been testing in hosting environments:

• KISS My Firewall: a free iptables script designed for a typical web server (stateful packet inspection, connection tracking, some preventative measures for port scanning, DoS attacks, IP spoofing, etc). It is one simply one file, can be installed with stock installations of Ensim WEBppliance Basic & Pro, Plesk, and Webmin, and automatically leaves open FTP, SSH, SMTP, DNS, HTTP, POP3, IMAP, HTTPS, MySQL, Secure IMAP, Secure POP3, Ensim WEBppliance Basic/Pro, Webmin, and Plesk. Open ports on the OUTPUT chain include: FTP, SSH, SMTP, RDATE, WHOIS, DNS, HTTP, HTTPS, and OPENSRS. A few quick changes, and you can quickly close any of the defaulted open ports.
• APF: Also IPtables-based and with an intelligent modular architecture and detailed usage information (made available with the apf command). (A great HOWTO here)
• BFD: a shell script that parses application logs authentication failures. (Another great tutorial)

My friend Jeff uses these successfully more in his hosting business with great success, and I’m curious as to what other similar tools admins use to standardize their LAMP boxes.

In some way, shape, or form, I’ve been following wireless for a while now. I got my amateur radio license and try to keep up with various spectrum legislation issues, particularly on where I feel our public commons are being threatened. For a while now, I’ve been wanting to shift into the wireless space, and the planets and stars seemed to be in the right order for this decision. In our team, there are approximately 4 “wired” (routing/switching/IP) and 2 “wireless” (RF/cellular), with varying degrees of overlap between all of us. Lento pero seguro, we’re each learning the other side of the fence…

On my first week, I had to revisit a few issues I rarely have to deal with in any great detail.

• RSTP (802.1w): these mesh boxes are, essentially, layer 2 devices with antennas. It’s not uncommon to see hundreds of these things, so spanning tree issues can be quite a problem. Each box has a tiny version of Linux, so from what I can tell, the potential to making switching decisions based on higher layers (7?) is there.
• MobileIP: how you keep a given IP address while roaming around your network.
• IGMP snooping: how our layer two doodads intercept IGMP messages update their MAC tables.
• L2TP: this helps “flatten” different networks for seemless mobility
• 802.11e (QoS): I expected this to be similar to 802.1P, but from what I can tell, it’s not a tag like it is in the wired world.
• 802.11r (roaming): a lot of 11e got rolled into 11r, and I’m still figuring out what goes into making a Wi-Fi <–> GSM handover possible.
• 802.11 in general: I’m quickly learning what’s part of the overall standard, and what’s determined by specific standards or vendor interpretation.

A good example of some of this coming all together is seeing how the sausage is made when it comes to seemless roaming. If you think of it in wired terms, major things go awry when one unplugs a client from one ethernet switch and plugs them in another one: MAC addresses are now associated with different switches, traffic that was en route now has try again, etc. Roaming in wireless is perhaps no different, and when something moves from one AP (access point) to another, traffic en route is disrupted similarly. I searched the 802.11 specification, but cannot seem to find anything that dictates how the client should associate and disassociate from each AP; each client card (firmware, software, etc) seems to dictate everything. One great analogy I’ve heard is that bugs (the “clients”) stick to light bulbs (the “AP”) until they die (i.e. signal fades away), then they begin to look for another AP to associate with.

Up to this point, I’ve pretty much taken for granted what vendors tell me about their IEEE implementation, either in their documentation or in their propaganda disguised as certification. In one week, I’m noted several interesting situations where vendors (not us, of course!) horribly simplify something (”seemless roaming”), is perhaps intentionally misleading (e.g. “full duplex wifi”), or implemented so poorly that it’s intended benefits are outweighed by some other crapiness (e.g. multiple radios in a box interfering with each other).

Today I happened to looked over my boss’ shoulder while he was configuring one of our wifi switch routers and noticed that he was using TFTPD32.

I downloaded it and was jazzed about seeing the DHCP, TFTP, SNTP and Syslog functionality that it has built in. Simply put, that means that when you reset a unit and it, say, defaults to DHCP mode, you can simply use this little exe program to quickly assign it an address, see the address that’s being assigned, and then rock and roll! Before this, I was doing all sorts of other craziness, such as nmap’ing the DHCP range of a network and then looking through the MAC addresses to find my unit.

You gotta be careful, though. Rumor has it that your computer supports RSTP and if you were, say, on a customer’s WiFi network *and* wired via ethernet some units (as our muni wifi boxes), you could run into problems and create some sort of bridging loop or something. (I have yet to confirm this and am still getting up to speed on RSTP issues in production).

TFTPD32 allows you to tweek tsize, blocksize and timeout, which (in theory) allow you to maximize performance when transferring large amounts of data. Haven’t tried this, but would be very curious if others here had…

You served me well, protecting me and often keeping me company during some cold nights in the data center. Remember that time I blasted you with the SmartBits because I doubted you? It seems like almost yesterday.

The only rocky point in our relationship was one armed routing, but looking back, it wasn’t wasn’t that bad, and you later worked out the kinks in 7.0. I’m sorry I later switched to CheckPoint. I hope you understand. I tried to still include you as much as possible, but CheckPoint’s friend, Nokia, just didn’t think you were cool enough to hang. Nokia brought up that one time when you crapped out when I pounded you with the SmartBits, and I didn’t stick up for you like I should have. I wish I was a bigger man then, but I wasn’t.

I like to think that you’re in a better place now.

Google Calendar integration with the iPhone is definitely a mixed bag: type in (something like) “lunch with friend noon today” in the “quick add” field, and it will automatically enter in the appropriate time; try to edit or delete that same entry within Google’s mobile pages, and it’s a no go.

It will be interesting to see how 4G drives IPv6 adoption.

(On OS X, I’ve been using Burn and have been quite impressed.)

Those wanting an open source alternative to Adobe and FoxIt Reader should check out Sumatra PDF. For only 500K, it packs quite a punch, particularly for lower end PCs that might need lighter tools.

(More tips like this can be found on the Adobe Gallery of Remedies.)

• change contact_groups from “admin” to “admins” (”admins” is probably already your default group in localhosts.cfg)
• define contact_groups in the host definition, rather than the hostgroup definition (based on the default config, this can be done by using “use linux-server” in your host definitions)
• edit address line (e.g. put in valid A record or IP address)

I threw a quick HOWTO on NagiosWiki, and eventually I hope to get to redoing that perl script to be compatible with 2.x/3.x cfg files.

• wget -r -l2 -t1 -nd -N -A.zip -erobots=off http://sdsickboy.com
• unzip ‘*.zip’

Now play the unzip’d .flac and .mp3 files in your music player.

• likes punk music,
• has a JD/esq (Escalante from Loyola; Orlando from UCLA),
• has a hispanic sounding last name,

and perhaps most importantly,

• likes to cautiously give half baked legal advice!

While I don’t know jack about podcasting, I urged him to consider creating a domain name and general theme, and once those were set, see what follows.

Mike is the type of dude who lives life in order to narrate it. I can only imagine what could happen if you gave him a soapbox, cheezy intro music, and some soft pitch legal questions on what the law says about building nonprofits.

• …can use F/OSS tools to build out crazy monitoring infrastructures (think tens of thousands of service checks).
• …is wanting to work in a fast-paced integrator environment (with sales people who find monster opportunities).
• …can communicate well (i.e. be able to respond to an RFP or create a solid Scope of Work).
• …is “plugged in” the F/OSS community (e.g. interacting with the people who wrote the source code for the tools which you’ll be using).
• …is adept at scripting and ok at programming (enough to read through and possibly modify other people’s perl and python scripts).
• …learn whatever technical skill they have to in order to “make it happen”.
• …can think creatively on how to solve “impossible” projects.
• …is able to interact comfortable with “c-level” executives (e.g. CTO, CIO, etc)
• …is willing to work in Newport Beach, CA.
• …is willing to travel once in a while.
• …can conduct [him/her]self like a professional.

Email me (Roger @ Hack My Idea . com) , we’ll chat on the phone, and I’ll give you the scoop and some (hopefully!) some helpful tidbits. After we talk, I’ll shoot your contact information and resume to the main decision makers.

I am NOT a headhunter, nor do I get any sort of financial compensation for this referral. I’m just looking to help an integrator I used to work for find a solid candidate. If nothing else and you’re not the best match, shoot me an email anyway, and maybe we can expand our list of professional contacts. If there is one thing in my career that I’ve learned, it’s that good people know other good people!

Next on my radar is a series of Cisco ASA HOWTOs (similar to the PIX series I wrote for Techsoup a few years ago). Compumentor just drop shipped me some equipment, so that should be cool…if for no other reason my professional contacts tend to be CheckPoint resellers!

For a while now, I’ve thought it would be cool to make an equivalent using sites like LightReading.com, Clay Shirky, etc.

I was quite pleased to stumble across this OPML file of all of NPR’s podcasts. (Here is another one on OPML.org that lists them in a more rational format.)

I went to CrazedList and put in one search term that spanned across all the relevant CraigsList sites (mostly sites in Southern CA, but some were nationally as well). I then put the CrazedList -generated OPML file in FeedMagick2’s “Inspect Pipeline: OPML Reading List Blender” and used the resultant RSS feed as my input value for Y! Pipe’s “Fetch Feed”. Using the “union” and “unique” operators, I could aggregate “Fetch Feed” inputs and parse out dupes.

(I probably spent 4 or 5 hours looking roll my own ghettofabulous ‘OPML to RSS’ parsing solution before throwing in the towel and just googling for an online tool to automagically do it for me. Please post any cool solutions you have in this area! Programming is not my primary skill, so go easy on me…)

Those wanting a “play by play” on how it works, can follow these steps.

1. Go to www.crazedlist.org and enter in your search string and terms (I’ll use “linux” as the search term, check “resumes”, and select “all” for regions)

2. Once we click the orange “get RSS feeds” button, we’ll get something like this.

3. Up near the top, you’ll see something like this

To get the OPML file right-click on this link to Save Link As…. when doing this you’ll be prompted to save a file called index.cgi or index.xml or something, you can change that and name it something like “mysearch.opml” then import that into your favorite RSS reader that supports importing feed list.

4. Open up FeedMagick2’s “OPML Reading List Blender” and enter in the path to the aforementioned OPML file. Convert that to RSS. (This may take a while). Note that path to that RSS feed. It will be your input for Y! Pipes.

5. Log in Y! Pipes, drag in the Fetch Feed module, and copy in that RSS feed. Link that to whatever modules you want, and publish your feed.

(Step 5 may seem unnecessary, but it shortens the path of the RSS feed considerably, making it more accessible to RSS readers which might not take such a long string)

While talking to my friend Chris about some ideas I had for Debian packages, he pointed me to a cool project called CheckInstall. This little tool looks awesome, especially considering I’m not that familiar with Debian’s way of doing things. Once your ‘make install’ is done, CheckInstall will create a Slackware, RPM or Debian compatible package and install it with Slackware’s installpkg, “rpm -i” or Debian’s “dpkg -i” as appropriate. According to the website, this script leaves copy of the installed package in the source directory so that you can install it wherever you want.

Hmmm…cool or dangerous? I’ll have to look at how exactly it operates and see if it really does everything necessary to make a “proper” .deb package.

This tutorial is a good quick overview on what it is and how it makes you more productive with multiple ssh sessions, particularly in environments where you need to collaborate with others, document your session, or make sure that a disconnection will not stop whatever process you’re running.

One frustration I have is not being able to go to the beginning of a line (control-a) while in screen. Any suggestions there would be greatly appreciated!

The Nagios class turned out to be quite a cool experience. I had about seven students who came from SAIC (1), Horizon Technology, (1), D-Link (4), and RK (1). LAMP skills between students varied widely, and I’m hoping that everyone left the class with something significant. Some of them have joined our LinkedIn group and have been networking with others in our group.

For what it’s worth, here are four more integration projects that I started this week. I will iron out HOWTOs on NagiosWiki once I get some time. (Those wanting specifics beyond what I have listed are welcome to email me, but please realize that I may not have all the kinks worked out yet)

1. ticketing and Nagios: once Nagios detects a down host or service, email OSTR (open source ticket request system) and autogenerate a troubleticket. When that host or service comes back online, the email on status change closes out the ticket. Once I iron out a few wrinkles, I’ll integrate this into RT (which I think is “better” in some ways).

2. Monitoring for ssh key corruption: When ssh keys get corrupted and need to be regenerated, check_ssh will not detect the login error (to my knowledge, at least). I’m not a Perl programmer, but I’m hoping that something in Net::Telnet (which can be told to use SSH for the underlying transport) or Net::SSH can help me prove that a login is failing on a few thousand routers. (Still googling for what others have done in this department. Any ideas here would be greatly appreciated!)

3. service reliability checks using NagiosPluginsNT: If you’d like to run a check *from* some weird nook and cranny in your network and do not want to deploy a Linux box with NSCA so you can relay passive checks, consider doing the following:

a. installing NSclient++

b. dropping the NagiosPluginsNT plugins in your c:\path\to\nsclient++\scripts directory

c. modding your c:\path\to\nsclient++\nsc.ini file to include

check_http_google=C:\Program Files\nsclient++\scripts\check_http.exe -H www.google.com

(Of course, test this from your Nagios server - “check_nrpe -H windows-box -c check_http_google“)

4. check_disk on /proc/mounts: started adding the following NRPE handler in the nrpe.cfg of various Linux servers with weird disk partitioning.

check command[check_disks_proc_mounts]=/usr/lib/nagios/plugins/check_disk -w 15% -c 10% $(for x in $(cat /proc/mounts |awk ‘{print $2}’)\; do echo -n ” -p $x “\; done)

(I had horrible problems with this command yesterday, as vim commented out certain sections, I wasted time trying to escape those characters. Muchos grassyass to my buddy Ed for helping me debug this one!)

Traditionally, one would just run “fdisk -l” or “df -h” and then write a separate NRPE handler for each command. In environments with crazy partitioning (or, better yet, NO partitioning!) or crazyass LUNs volumes, you gotta just send one command that checks the collective health of everything and then reports back if one of those volumes has exceeded its critical or warning level. If that server is important enough to check a particular volume or media for a specific parameter, then consider hard coding a specific NRPE handler for that server.

5. DNX + Nagios: This project offloads active checks to worker boxes, saving you (theoretically) lots and lots of time changing active checks to passive ones via NRPE and NSCA. I just untar’d the project and have been reading over the documentation. It looks easy enough to integrate, but I’ll know more when I put the rubber to the road.

The bottom line to these projects: automate, automate, automate! If you have to do it once, then do it manually. If you have to do it twice, do it manually *and* document. If you have to do it manually a hundred times? Nigga please….automate, yo!

I found fping in these situations to be the “magic bullet” to help me start narrowing down DNS problems.

e.g. something like: fping -d -g 10.0.0.0/24

-g would let me feed in a weird subnet (or range of IPs), and -d would turn around and ping the host name that the ICMP reply gave me, ultimately giving me the A record someone needed to BIND to help with the transition.

From there, I could use the standard unix tools (grep, sed, etc) to sort whatever output fping crapped out.

Anyone have a “better” way of doing this?

(Just who do they think that their market is with that description?)

(1) grab the pre-compiled binary, unzip, and then untar it in the /opt directory, which is where many admins like to store large packages.

cd /opt
wget http://www.domain.com/path/to/nrpe-nsca-plugins.tgz
gunzip /opt/nrpe-nsca-plugins.tgz
tar xvf /opt/nrpe-nsca-plugins.tar

(those who’d like to simply look at the contents of the tar can type tar tvf)

(2) Now grab and configure the /etc/ files. Some admins (like myself) like all the /etc/ config files to be in one location.

cp /opt/nagios/etc/* /etc/

OR log on another similar box and type in the following to push it over to this box

scp nrpe.cfg root@otherbox:/etc

OR grab nrpe.cfg from another box and copy to your /etc directory

scp root@otherbox:/etc/nrpe.cfg /etc/

(3) modify the NRPE handlers in nrpe.cfg to include the correct path to the binaries you gunzip’d and untar’d in /opt

command[check_users]=/opt/nagios/libexec/check_users -w 5 -c 10

(4) now, add Nagios’ NRPE to the /etc/rc.local for each reboot

nohup /usr/bin/nrpe -c /etc/nrpe.cfg -d

(5) Now, start the NRPE service and associate it with the /etc/nrpe.cfg file you cp’d over

/usr/bin/nrpe -c /etc/nrpe.cfg -d

(6) Now, double check everything to make sure it’s all working. That includes: (a) checking the running processes on your AIX server, (b) checking the rc.local file on your AIX server, and also (c) making sure that your Nagios server can access the AIX server using its check_nrpe plugin. If there is a problem, try tail-ing the syslog for clues (tail /var/adm/syslog)

less /etc/rc.local
nohup /usr/bin/nrpe -c /etc/nrpe.cfg -d > /dev/null 2>&1 #Start NRPE for Nagios

AND

ps -ef | grep nrpe
root 458886 589858 0 01:11:51 pts/0 0:00 grep nrpe
nagios 536778 1 0 01:11:13 - 0:00 /usr/bin/nrpe -c /etc/nrpe.cfg -d

From Nagios (this was on my CentOS 4.x box)

cd /usr/lib/nagios/plugins/
./check_nrpe -H AIX_Server -c check_whatever
(RETURN OK)

Thanks to Everyone Who Pitched in to Keep Our Podcast Free

A few weeks back, we asked you to help our home station pay the substantial bill for offering our show as a free podcast and online stream. A huge thank-you to the thousands and thousands of people who contributed: in a few, short weeks we raised enough to cover all the bandwidth we used in 2007. All of us at This American Life and Chicago Public Radio thank you, again, for your generosity!

Of course, it’s always been free to the technorati who have know the secret URI to download whatever episode they’ve wanted. To find it, fire up Wireshark, start it on the interface that is connected to the Internet, and you will see a GET command in the INFO field similar to the following:

HTTP GET /jomamashouse/ismymamashouse/(EpisodeNumber).mp3 HTTP/1.1

So, in layman’s terms, you can simply put the following URI in your browser window to download and listen to past episodes!

http://audio.thisamericanlife.org/jomamashouse/ismymamashouse/(EpisodeNumber).mp3

(Or, if you have Linux or OS X, go to the command line and put a wget before the URI. A win32 version of wget can be found here)

Several months ago, my friend Chris at LSU’s SRCC recommended me lftp. I’ve used it quite extensively on Linux servers. It’s a wonderful quick and dirty cli tool, particularly when quickly scripting out FTP PUTs or GETs on Windows servers is involved. (Here is an excellent tutorial for newbies)

Today I found a Windows binary of LFTP on the Doom9’s forum. I scanned it with up-to-date AVG, ClamAV, and Symantec signatures, and it all seems to be kosher.

Those wanting to see how it works before they download it might appreciate the following lftp –help output.

lftp –help

Usage: lftp [OPTS]

`lftp’ is the first command executed by lftp after rc files
-f execute commands from the file and exit
-c execute the commands and exit
–help print this help and exit
–version print lftp version and exit

Other options are the same as in `open’ command
-e execute the command just after selecting
-u [,] use the user/password for authentication

-p use the port for connection
host name, URL or bookmark name

Personally, I do it all with the following type of command:

lftp ftps://user:password@ftp.yourserver.com:990

(From there, you can quickly navigate through the folders like you’d expect to normally in in FTP.)

Those paranoid about installing anything on a stable box should probably closely examine the hashes and contents of the install.bat file.

lftp-install.zip ea72152c91b52ac7ae5f962764eef4eb

install.bat

copy *.* %windir%\system
regsvr32 %windir%\system\cygwin1.dll
regsvr32 %windir%\system\cygncurses5.dll

lftp.exe 8be4378428af8a18c9d1cc58abac1241 (md5)
cygncurses5.dll 2044dbb9e65b51d5345be7d62b27477e (md5)
cygwin1.dll 596938bda60e655b82a68adb315e3bc6 (md5)

To see what he was made of, I simply asked him if he ever had to observe the 5-4-3 rule. He had never heard of it. And when I told him what it was, he then told me that was no longer needed. And when I told him that to my limited understanding it was needed “back in the day”, he changed the subject to vampire taps! Hmmm…go figure. (So maybe it wasn’t just me when we seemed to have a little disconnect between the difference between a broadcasting and a collision domain.)

Hands down, my favorite posers thus far are the “forensic experts” I’ve screened. I don’t know beans about forensics, but I have a clue when it comes to certain key concepts, and if pushed, I could fight my way out of a paper bag on any of those topics, as well. One “expert” said that he had all of these forensic skills, and when I finally asked about how exactly he acquired the data before he analyzed it (one of the few forensic topics on which I’m somewhat knowledgeable), I got an answer on how he didn’t do it. When I pushed him on the nitty gritty on how he “forensically searched” the contents of people’s copmuters, I found that he had done little more than install EnCase on his computer and conducted extremely rudimentary searches on some EnCase images.

Being a sort of technology generalist, I can certainly understand that no one person can understand everything about a given subject. It’s often easy to get caught not knowing very basic things about something you’re supposedly an “expert” in. But when it’s obvious that you’re batting WAY out of your league, c’mon…

Here is a little command that helps when you must mass move tons of files from one NT box to another. These funky switches are important, as they help grab all the subdirectories, hidden junk, and meta-crap.

xxcopy x:\folder1 y:\folder2 /s /h /tca /tcc /tcw

If I’m feeling bitchy about hitting Y too many times, I like add the /yy command. At the end you’ll have a nice little report that tells you what hiccuped.

Often times I’ll find some cool little hack tool to run against a bunch of files, and that little hack tool does have good folder recursion features.  A quick kludge in these cases is to just have xxcopy “flatten” all the directories in the destination folder and then let my hack tool go to town!

1. Have all sorts of weird aliases, such as info@domain.com, etc.
2. Use Exchange 5.5/2000/2003 (none on 2007 yet)
3. Use different clients to access their mail (making it difficult to quickly solve the problem via one of my favorite Outlook spam plugins, Cloudmark)
4. Have a Crackberry and/or iPhones (which beeps incessantly because of all the spam).
5. Refuse to immediately buy a real solution
6. Are almost solely responsible for the server queue with thousands of spam messages, which of course screw up Exchange’s Information Store jeopardizes the health of the mail server. (And me constantly running to the Exchange server and to type (aqadmcli and then delmsg flags=all is NOT a solution just because it “worked” last time)
7. Don’t want Linux in their organization and/or don’t have a spare server for me to throw Postfix on.

So, as long as the email sent is some sort of alias (and not their user name), you can often just kludge the following solution:

1. Create an Exchange forwarder for that alias@domain.com address
2. Setup Gmail to relay back to the internal address of the Exchange server.

Ghettofabulous, I know. But sometimes that’s how you gotta roll!

Everyone is worried about losing email in this process, but honestly, the biggest risk in this is it working so well for these VIP types that they want you to now do that hundreds of times for all the other users!

File before: 3,469 KB
(File after: 1,473 KB)
—————————————————
crap no one needs:  1,996 KB

So, there’s this huge push to make sure (a) stuff says on NTFS drives (and doesn’t wonder to FAT32 drives), or (b) someone doesn’t boot to a live cd and bypass NTFS permisssions, but what happens within NTFS and Active Directory to make sense of the madness? I’ve done a lot with Active Directory, and every time The Big Cheese asks me who can access what, I cringe. Doing it within Windows can sometimes be quite a chore, and even when I use third party tools such as AccessEnum to make that process less painful, it’s still sometimes a big chore. I can remember a couple of years ago having to nitpick NTFS permissions for a biomedical client of mine who was paranoid about unauthorized document changes on engineering specs and having to set up groups / users and then use all sorts of auditing tools to make sure that I (or another admin) didn’t give too many permissions to various users.

One solution to this scenario might be with ‘data governance’ tools like Varonis. Using their solution, you can do lots of things that’s a HUGE pain in the ass with native Active Directory (AD) tools, such as creating users and permissions for specific time periods, creating changes in a sort of AD sandbox, finding dependencies on AD objects, etc. I met with the channels rep from Varonis the other day and was quite impressed with their demo. Using their product, someone in a business unit could, say, add new permissions for contractors on special projects, and once x out of y people approved that addition, the contractor would have AD permissions for Z amount of time. Or, say that someone needed to clean up AD (something that’s always a pain in big organizations), they could turn on Varonis’ reporting (which is way less resource intensive than Windows’ native file auditing, which sort of assumes you know files or users are suspect). Lots of cool stuff there….

I was impressed with how many bases Varonis covers. From a C-level perspective, it puts business units in the driver’s seat. In too many companies, IT departments (for good or bad reasons) cripple business functionality. From a tech “trenches” perspective, I’ve now got a tool that helps me do easily what was previously quite arduous and take care of duties that would otherwise have fallen in the cracks.

Pricing on Varonis is a sort of combination between these probe clients installed on file servers, as well as AD users. As you might guess, it is insanely expensive for the the average company, leaving it up to places that are willing to pay huge premiums to ensure that they’re SOX compliant.

(Soon I should have a demo running on my VMware server, and perhaps can have a better review at that time.)

While I haven’t yet used Ruby-based AutomateIT in production, I wonder if it (or something like it) will be soon be key to easily automating tools like Procmon and documenting their effects on Windows systems (e.g. DLL calls, registry hooks, and dependent processes). Server automation tools (e.g. CFEngine, Puppet, etc) are key for many business reasons — reducing risk / errors / downtime, simplifying updates / migrations / recovery, codifying knowledge into repeatable “recipes” (sudo make me a sandwich!) — but perhaps soon we might see them integrated into incident response toolkits?

I don’t see a demo on their site, but I do see copies on YouTorrent and Torrentz. (Not sure if they’re authorized copies, though)

Because of a screwy cd drive on the laptop, I couldn’t just boot into the Sid / Knoppix-based Helix CD and use LinEn to image, but rather had to remove the hard drive, break out my newly purchased UltraKit write blocker kit, and image it that way (which, while more forensically sound, takes much longer to do).

Overall, the Ultrakit is fairly straight forward. In one side of the Tableau IDE write blocker, you plug in an IDE cable, which plugs into the IDE converter for your little IDE laptop hard drive. In the other end, you plug in your A-A USB cable, which plugs into your computer’s USB port. In the computer’s other USB port, you plug in your target USB drive (a 500 GB Seagate, in my case). Lights on the Tableau clearly tell you that everything is working ok (disk activity, connection to host, write blocking enabled, power, etc).

About an hour into imaging the laptop hard drive, I decided to kill the job and start the process on a laptop that was significantly faster. Bad idea — something I should have know better than do! Unplugging and rebooting Helix must have resulted in some sort of dirty dismount, because after booting on the new laptop and mounting the same Seagate 500 GB hard drive with ntfs-3g, I couldn’t seem to write properly to the drive without some sort of “input/output” error. Instead of using Sid’s ntfstools to “fix” the problem (which I’ve done in the past), I just used an XP Pro workstation to quickly format it again.

I then tried to use my Intel-based Macbook Pro for the imaging, but for whatever reason, Helix wouldn’t boot past the basic Grub menu. Instead of trying to figure out why, I just finished the job on my HP laptop.

For those new to the process, here are some steps I did to image the drive (most importantly, I guess, is NOT to mess with the connection once you start the image transfer!).

(1) Quickly format the destination USB hard drive to NTFS. Since I don’t completely trust the NTFS tools, I most always use Windows. (One recommended step before this is to zero out the hard drive in Helix by typing [something like] dcfldd if=/dev/zero of=/dev/sda bs=8k conv=noerror,sync, of if you want to securely wipe it, type something like wipe -kq /dev/sda)

format e: /fs:ntfs /q

(2) Once in Helix, check out which devices the operating sees (but may not have mounted).

fdisk -l

(3) Say it’s /dev/sda (I sure hope so, or you’ve zero’d out the wrong hard drive!), let’s make sure that it’s completely dismounted. If not, then we will need to do close or kill anything else in Helix which may be accessing that USB drive.

umount /dev/sda1

(4) Create a folder that we can later mount to, as well as a folder within it that we can put in the forensic images we get from LinEn.

mkdir /mnt/bigUSB
mkdir /mnt/bigUSB/forenicImages

(5) Now, let’s map that USB device in /dev to the aforementioned folders.

ntfs-3g /dev/sda /mnt/bigUSB

(6) Once you mount that folder, you are ready to acquire an image with LinEn. Start it and select the /dev devices which you are imaging (not the hard drive that will be written to). Now, when you enter your save path, enter:

/mnt/bigUSB/forensicImages

(7) Enter in all your forensic notes, and then sit for a while.

Le voila! On fairly mediocre laptop, it took about an hour for every 20GBs. Afterwards, you can use the hashing tool to make hashes of the attached hard drive to ensure that nothing was lost or altered in your images.

While I used NTFS (because I had some ~50 GB EDB file), it’s important to note that Guidance Software, the makers of Encase, suggest using FAT32 on the target drive (which can be done by typing something like mount -t vfat /dev/sdb1 /mnt/bigUSB). I suspect that this is a result of portability, as FAT32 is a sort of “lingua franca” when it comes to file systems. Also of interest to forensic types is the wide array of other tools out there for making Encase images, some of which claim to make Encase images faster than LinEn.

Update: One extra step you should perform when doing forensics is computing and writing down the hash of the entire partition (this, of course, assumes that sdb [2nd USB drive] is your ’suspect’ drive, not your target drive).

e.g.

md5sum /dev/sdb
sha1sum /dev/sdb

One of his tactics is to pick particular events (which, if taken into context, are probably what any reasonable person would have done) and then hammer away on people. By selectively picking something “stupid” and constantly revisiting that “stupid” mistake, he successfully beats others into submission so that they are less likely to resist in the future. He bolsters his legitimacy to do so by claiming that president of the company holds him personally accountable for everything under him.

In light of this mess, I couldn’t help but mass send this Frontline PBS episode to most everyone in the department. My favorite is the end (clips 4 and 5), where the correctional officers are put through the same exercises as the 3rd graders. In situations like this, the “inferior” group rarely (if ever) wins. If they argue, then they’re exhibiting the “inferior” traits that they supposedly have, and none of the others in the group will do anything but sit on the side lines and hope to eventually be a part of a winning team.

Perhaps the most bitter pill in all of this is that in the end, this yahoo isn’t doing anything that his employees aren’t letting him do.

I have no idea if the following video below is staged or not (why are the main people mic’d?), but it captures the build up of these tantrums well: someone is late to some sort of conference (which seems to always be held in a hotel), the main sales / biz dude gets really restless and starts making a scene (on little things like empty coffee pots), and then starts slinging things and barking at random people.

I’ve seen this multiple of times in the technology industry with various sales people, and for whatever reason, these yahoos still moving up the food chain. And when you talk to someone at a different company who worked with them, it’s always great to compare their tantrum stories!

Enjoy!

CCIE Lab Strategy - A Structured Approach

Pros to such a stunt:

• You finally get to stick it to your pointy haired boss.
• Your become hero to other downtrodden employees.
• Will live long in company folklore.
• The more “evil” the company, the better you feel after doing it.
• If your antics put your boss on tilt, you win.
• You embolden other coworkers who might also feel shat on.
• People like me will blog about you.

Cons against such a stunt:

• Possible fallout with coworkers who have to pick up your slack.
• Coworkers might have to now be careful about saying that they associate with you.
• Others who hear of your antics might not understand (or care to understand) context.
• Severe business interruption if what you do is important.
• Even more business interruption if others take your lead
• Unnecessarily escalating a situation

I’ll have to admit being somewhat amused by this antic. Sticking it to someone always feels good, but in the end, this sort of action carries way more liability than it is probably worth. Also, people who are incorrigible in one context might not be so bad in another (and vise versa). So, your pointy haired boss today might actually be a good trench mate at your next company, so why rock the boat unnecessarily? In fact, some of my best friends and associates today are peope who I butted heads with in the past.

From my experience, the best thing to do is just find a better opportunity, leave on good terms, and then resell services back to the company. If you play your cards right, you can sell services back to the company and make almost the entire amount you made before at a tiny fraction of the effort you had to expel before.

If you’re the boss, it’s important to remain un-phased by these antics. If I were the recipient of such a stunt, I’d probably just reply with the following pic, along with a note saying that things wouldn’t be the same without him or her!

My dear coworkers, do yourself a favor and drop Rand’s “A is A” drivel and 2 dimensional mega-characters and pick up some real libertarian thinkers, like Robert Nozick or Noam Chomsky. You (and the others in the offices who are also sick of hearing about your thoughts on Rich Dad, Poor Dad) will thank me later.

(Found the vid while scanning through Cisco and CCIE tags on del.icio.us)

Download and untar:

cd /tmp
wget http://freshmeat.net/redir/nagioscheckbl/58783/url_tgz/nagios-check_bl-1.0.tar.gz
tar zxfv nagios-check_bl-1.0.tar.gz


Copy perl script in your Nagios plugins directory

cd nagios-check_bl-1.0
cp check_bl /usr/lib/nagios/plugins/


Test plugin using your mail server and, say, zen.spamhaus.org

cd /usr/lib/nagios/plugins/
./check_bl -H mail.yourdomain.com zen.spamhaus.org


(If get weird error, then perhaps something relating to Perl-Net-DNS not being installed)

yum install perl-Net-DNS-*


Finally, add a command definition in /etc/nagios/commands.cfg so you can call it in services.cfg (I add as many blacklists as possible)

#tested on CentOS 4.x
define command {
command_name check_bl
command_line $USER1$/check_bl -H $HOSTADDRESS$ -B zen.spamhaus.org bl.spamcop.net dnsbl.ahbl.org dnsbl.njabl.org dnsbl.sorbs.net virbl.dnsbl.bit.nl rbl.efnet.org phishing.rbl.msrbl.net 0spam.fusionzero.com list.dsbl.org multihop.dsbl.org unconfirmed.dsbl.org will-spam-for-food.eu.org blacklist.spambag.org blackholes.brainerd.net blackholes.uceb.org spamsources.dnsbl.info map.spam-rbl.com ns1.unsubscore.com psbl.surriel.com l2.spews.dnsbl.sorbs.net bl.csma.biz sbl.csma.biz dynablock.njabl.org no-more-funn.moensted.dk ubl.unsubscore.com dnsbl-1.uceprotect.net dnsbl-2.uceprotect.net dnsbl-3.uceprotect.net spamguard.leadmon.net opm.blitzed.org bl.spamcannibal.org rbl.schulte.org dnsbl.ahbl.org virbl.dnsbl.bit.nl combined.rbl.msrbl.net
}


I currently have this checking about 400 servers, and it seems to be doing a fairly good job!

(1) see the big picture, and (2) come through on the details

Lots of people can do (1), and lots of people can do (2); but few people really can do the right combination of (1) and (2). (I know lots of brilliant technology professionals I know can do both (1) and (2), but, for whatever reason, cannot properly articulate that in a resume, and that puts them at a huge disadvantage to the people out there who can talk the talk and never get called on walking the walk)

One technique I use to assess a candidate is to often just hand them a piece of paper and have them draw out the “biggest” or “last” network that s/he has ever supported / built / maintained. Seeing how and what they draw out gives me a good idea on where they feel comfortable, then I start to drill down to see what they know best and how they improved things at their company. This often gives me a good idea of the business decisions that drove what the person did (or didn’t do) there.

To be honest there is only one conclusion to be made; Microsoft have really outdone themselves in delivering a brand new operating system that really excels in all the areas where Vista was sub-optimal. From my testing, discussions with friends and colleagues, and a review of the material out there on the web there seems to be no doubt whatsoever that that upgrade to XP is well worth the money. Microsoft can really pat themselves on the back for a job well done, delivering an operating system which is much faster and far more reliable than its predecessor. Anyone who thinks there are problems in the Microsoft Windows team need only point to this fantastic release and scoff loudly.Well done Microsoft!

I usually wait til SP1 before I start using a new MS product. In this case, I’m going to be waiting a lot longer….

• Onaro’s VM Insight
• Marathon Technology’s everRun FT for Xenprise
• InovaWave’s VirtualOctane for ESX Server
• IBM’s IBM System x 3950 M2 with VMware ESX Server 3i (an IBM data-class machine with four quad-core processors in one frame, connectable via InfiniBand, running an embedded hypervisor)
• CiRBA’s Data Center Intelligence 4.4
• 3Leaf Systems’ V-8000 Virtual I/O Server
• Akorri’s BalancePoint 1.7
• B-hive Networks’ Conductor, Netuitive’s SI for VMware
• PlateSpin’s PowerConvert, Leostream’s P>V 4.0
• Symantec NetBackup
• Blue Lane’s VirtualShield
• LeftHand’s Virtual SAN Appliance
• Veeam Software’s Veeam Reporter 2.0
• ThinPrint GmbH’s .print Virtual Desktop Engine
• ClearCube’s Sentral 5.6
• VMware’s VMware Workstation and VMware ACE 2

I will definitely have to check out a few of these vendors in 2008, particularly Akorri.

• Config file is written in Ruby
• Easily write your own custom conditions in Ruby
• Supports both poll and event based conditions
• Different poll conditions can have different intervals
• Integrated notification system (write your own too!)
• Easily control non-daemonizing scripts

Nagios::Plugin::Getopt is an OO perl module providing standardised and simplified argument processing for Nagios plugins. It implements a number of standard arguments itself (–help, –version, –usage, –timeout, –verbose, and their short form counterparts), produces standardised nagios plugin help output, and allows additional arguments to be easily defined.

It consists of

Nagios::Plugin::Getopt

Nagios::Plugin: A family of perl modules to streamline writing Nagios plugins
Nagios::Plugin::Config: read nagios plugin .ini style config files
Nagios::Plugin::ExitResult: Helper class for returning both output and return codes when testing.
Nagios::Plugin::Functions functions to simplify the creation of Nagios plugins
Nagios::Plugin::Getopt: OO perl module providing standardised argument processing for Nagios plugins
Nagios::Plugin::Performance: class for handling Nagios::Plugin performance data.
Nagios::Plugin::Range: class for handling Nagios::Plugin range data.
Nagios::Plugin::Threshold class for handling Nagios::Plugin thresholds.

Ok, maybe it’s not in the same league as some of the others, but it’s a godsend to anyone who (like me) regularly inherits horribly fouled up Windows servers. (My favorite is still the accounting server that also had ASSP running on it. I scanned it with an AV scanner and found thousands of viruses! I assure you, Mr. MCSE, I have not forgotten your little faux pas.)

While it’s still something a little fuzzy in my mind, it gives me a quick idea on where that candidate stands on several levels:

• What distros does this candidate know and prefer?
• How deep do they think about things that they might do every day routinely?
• How well can they describe things?
• Do they change their mind once I correct them with a “fact” that is obviously wrong?
• What areas are they most comfortable talking about here? e.g. BIOS, hard drive boot sector, kernel boot, system init, start up scripts, etc?
• How easily can they possibly repair a crashed system?

Supports multiples tsm servers (servername)

Can be used transparently as a nagios plugin

Alert notification mechanism (by e-mail)

Supports new values for ok/warning/critical status in command line

Bourne shell (sh) compliance

Easy to add news checks

Here are some more examples of its coolness:

———————————————————————
show all checks helpUsage..: tsmmonitor help
Example: tsmmonitor help
———————————————————————
check tsm database utilization

The default percentages are:
warning..: 85
critical.: 90

Usage..: tsmmonitor db [warning] [critical]
Example: tsmmonitor db
tsmmonitor db 80 95
———————————————————————
check tsm recovery log utilization

The default percentages are:
warning..: 60
critical.: 80

Usage..: tsmmonitor log [warning] [critical]
Example: tsmmonitor log
tsmmonitor log 80 95
———————————————————————
check scratch tapes minimum number

The default numbers are:
warning..: 10
critical.: 6

Usage..: tsmmonitor scratch [warning] [critical] [library_name]
Example: tsmmonitor scratch
tsmmonitor scratch 8 4
tsmmonitor scratch 8 4 LTOLIB3
tsmmonitor scratch LTOLIB3
———————————————————————
check number of drives not online

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor drive [warning] [critical] [library_name]
Example: tsmmonitor drive
tsmmonitor drive 2 3
tsmmonitor drive 1 2 LTOLIB3
tsmmonitor drive LTOLIB3
———————————————————————
check number of paths not online

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor path [warning] [critical]
Example: tsmmonitor path
tsmmonitor path 2 4
———————————————————————
check tsm database fragmentation

The default numbers are:
warning..: 60
critical.: 80

Usage..: tsmmonitor dbfrag [warning] [critical]
Example: tsmmonitor dbfrag
tsmmonitor dbfrag 50 75
———————————————————————
check number of unavailable volumes

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor unav [options] [warning] [critical] [device_class]
-v, show unavailable volumes
Example: tsmmonitor unav -v
tsmmonitor unav 2 4
tsmmonitor unav 2 4 LTOCLASS
———————————————————————
check number of pending requests (query request)

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor req [warning] [critical]
Example: tsmmonitor req
tsmmonitor req 2 3
———————————————————————
check a storage pool utilization

The default numbers are:
warning..: 80
critical.: 95

Usage..: tsmmonitor stgpool [warning] [critical]
Example: tsmmonitor stgpool DISK_POOL
tsmmonitor stgpool DISK_POOL 50 75
———————————————————————
check for volumes with error (error_state)

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor volerr [options] [warning] [critical] [device_class]
-v, show volumes with error
Example: tsmmonitor volerr
tsmmonitor volerr -v 3 5
tsmmonitor volerr 3 5 LTOCLASS
———————————————————————
check how many tapes are in the library

The default numbers are:
warning..: 90
critical.: 86

Usage..: tsmmonitor tapeslib [warning] [critical] [library_name]
Example: tsmmonitor tapeslib
tsmmonitor tapeslib 120 115
tsmmonitor tapeslib 120 115 LTOLIB3
tsmmonitor tapeslib LTOLIB3
———————————————————————
check how many tapes have a specific owner

The default numbers are:
warning..: 2
critical.: 3

Usage..: tsmmonitor tapesown [warning] [critical]
Example: tsmmonitor tapesown tsmsrv01
tsmmonitor tapesown tsmsrv01 4 5
———————————————————————
check how many volumes are in a specific storage pool

The default numbers are:
warning..: 40
critical.: 50

Usage..: tsmmonitor tapesstgpool [warning] [critical]
Example: tsmmonitor tapesstgpool DAILY
tsmmonitor tapesstgpool DAILY 30 45
———————————————————————
check how many tsm db backup there are in the last 24 hours

The default numbers are:
warning..: 1
critical.: 0

Usage..: tsmmonitor dbbkp [options] [warning] [critical]
-v, show some informations about database backup
Example: tsmmonitor dbbkp
tsmmonitor dbbkp -v
tsmmonitor dbbkp 2 1
———————————————————————
check number of nodes sessions

The default numbers are:
warning..: 15
critical.: 20

Usage..: tsmmonitor numsess [warning] [critical] [session_state]
Example: tsmmonitor numsess
tsmmonitor numsess 20 30
tsmmonitor numsess 20 30 MediaW
tsmmonitor numsess Run
———————————————————————
check number of nodes

The default numbers are:
warning..: 80
critical.: 90

Usage..: tsmmonitor numnodes [warning] [critical] [domain]
Example: tsmmonitor numnodes
tsmmonitor numnodes 20 30
tsmmonitor numnodes 20 30 SAP
tsmmonitor numnodes SAP
———————————————————————
check number of disk volumes without readwrite access

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor diskvol [options] [warning] [critical]
-v, show volumes without readwrite access
Example: tsmmonitor diskvol
tsmmonitor diskvol -v
tsmmonitor diskvol 2 3
tsmmonitor diskvol -v 2 3
———————————————————————
check number of database volumes not synchronized (copy status)

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor dbvol [warning] [critical]
Example: tsmmonitor dbvol
tsmmonitor dbvol 2 3
———————————————————————
check number of log volumes not synchronized (copy status)

The default numbers are:
warning..: 1
critical.: 2

Usage..: tsmmonitor logvol [warning] [critical]
Example: tsmmonitor logvol
tsmmonitor logvol 2 3
———————————————————————
check server license compliance

Usage..: tsmmonitor lic
Example: tsmmonitor lic

Check out the Nagios plugin, check_logfiles. From their website:

MotivationThe conventional plugins which scan log files are not adequate in a mission critical environment. Especially the missing ability to handle logfile rotation and inclusion of the rotated archives in the scan allow gaps in the monitoring. Check_logfiles was written because these deficiencies would have prevented Nagios from replacing a propritetary monitoring system.

Features

• Detection of rotations - usually nightly logfiles are rotated and compressed. Each operating system or company has it’s own naming scheme. If this rotation is done between two runs of check_logfiles also the rotated archive has to be scanned to avoid gaps. The most common rotation schemes are predefined but you can describe any strategy (shortly: where and under which name is a logfile archived).
• More than one pattern can be defined which again can be classified as warning patterns and critical patterns.
• Triggered actions - Usually nagios plugins return just an exit code and a line of text, describing the result of the check. Sometimes, however, you want to run some code during the scan every time you got a hit. Check_logfiles lets you call scripts either after every hit or at the beginning or the end of it’s runtime.
• Exceptions - If a pattern matches, the matched line could be a very special case which should not be counted as an error. You can define exception patterns which are more specific versions of your critical/warning patterns. Such a match would then cancel an alert.
• Thresholds - You can define the number of matching lines which are necessary to activate an alert.
• Protocol - The matching lines can be written to a protocol file the name of which will be included in the plugin’s output.
• Macros - Pattern definitions and logfile names may contain macros, which are resolved at runtime.
• Performance data - The number of lines scanned and the number of warnings/criticals is output.
• Windows - The plugin works with Unix as well as with Windows (e.g. with ActiveState Perl).

• Awk one liners
• Sed one liners

WRONG! “TTL reply” in ping is completely different than the “TTL in transit” in traceroute. Traceroute sends out an ICMP echo packet to a host, but with a TTL of 1 (then TTL 2, TTL 3, etc). Once the host gets the standard ICMP ‘echo rely’ packet back from the final host, traceroute gets a ‘TTL expired in transit’ message back.

(1) List the to make sure that they all are there ok:

#ls
1.htm 2.htm 3.htm 4.htm

(2) Now, we’ll use awk to print out “mv ” plus the results of each colume (in this case, a column equals a file) two times with a space in between, the second time adding an “L” to the end of the output.

#ls | awk '{print "mv "$1" "$1"l"}'
mv 1.htm 1.html
mv 2.htm 2.html
mv 3.htm 3.html
mv 4.htm 4.html

(3) Once that looks good, we’ll tack on a pipe and the ’sh’ command in order the command line to execute that script.

#ls | awk '{print "mv "$1" "$1"l"}' | sh


Le voila! All those files are changed.

Now, say that they want the opposite? This gets a little tricky just using awk because we’re deleting a character as opposed to simply adding, so we’ll add some of unique garbage to the end of the first file, then use sed to search and replace that garbage string.

(1) List all the files in the directory.

#ls
1.html 2.html 3.html 4.html

(2) Now, we do something very similar to our previous step using awk, except this time, we put on some trash string at the end in order to uniquely idendify it later when we do a search and replace.


#ls | awk '{print "mv "$1" " " "$1".ZZZ"}'
mv 1.html 1.html.ZZZ
mv 2.html 2.html.ZZZ
mv 3.html 3.html.ZZZ
mv 4.html 4.html.ZZZ

(3) Perfect, now we can replace the ‘html.ZZZ’ ending with our desired ‘htm’ one instead

#ls | awk '{print "mv "$1" " " "$1".ZZZ"}' | sed 's_html.ZZZ_htm_'
mv 1.html 1.htm
mv 2.html 2.htm
mv 3.html 3.htm
mv 4.html 4.htm

(4) Like before, once the output looks good, we’ll tack on ” | sh” to the end to tell the computer to run it as a special command.

ls | awk '{print "mv "$1" " " "$1".ZZZ"}' | sed 's_html.ZZZ_htm_' | sh

Update: Sr. Unix Pimp, Jarvis Talley, just wrote me with a more elegant solution:

mkdir /tmp/htm_files
grep htm * | grep -v html | cut -d : -f 1 | xargs -t -i cp {} /tmp/htm_files.and

for i in `cat /tmp/htm_files`
do
cat $i | sed ’s/htm/html/g’
done

(I’ll most likely use this in conjuction with ‘nagiostats’ and sed/awk to help me determine when Nagios checks are lagging and needs to be seriously ‘re-thunk’ — e.g. make checks passive [instead of active], reduce frequency of checks, install DNX, build entirely separate Nagios server, etc)

BalancePoint’s Cross-Domain Analysis™ technology collects information from servers, storage, and infrastructure software, and automatically correlates application performance across these domains. By providing a single view across IT silos, Cross-Domain Analysis shortens troubleshooting, saves money, and helps prevent disruptive outages to business-critical applications.BalancePoint’s Application Fingerprinting™ technology characterizes application workloads and analyzes the infrastructure’s ability to service it.

It can even predict the future. BalancePoint’s advanced analytics and modeling can give early warning to potential application performance problems before they happen. The system can help companies understand their storage capacity requirements so they can plan and manage purchases well in advance.

BalancePoint is fully “virtualization aware”. It supports many storage virtualization platforms and was designed for VMware. Akorri is a VMware Technology Alliance Partner.

IT Operations and Infrastructure managers, system/storage administrators, and application owners all benefit from BalancePoint’s ability to:

* Plan for growth and change, getting actionable provisioning recommendations that account for the impact on application performance.
* Optimize application performance and resource utilizations. Automatically find deeply buried contention, hotspots and bottlenecks.
* Manage performance across applications, servers and storage with cross-domain visualization and performance-based service alerting.

When problems are found get direct automatic troubleshooting analysis.

Too good to be true? Well, I’ll know more when the sales reps responds to my sales inquiry…

When that’s not feasible, I just quickly Monit. As you can see from these examples, babysitting Nagios and Cacti dependencies (e.g. httpd, sshd, etc), alerting when there is a problem, and event handling can be extremely easy. If you would like to make absolute sure that you’re running Monit on your system, you can control it via init in order to have init automatically spawn another daemon.

e.g.
/etc/inittab:

# Run monit in standard run-levels

mo:2345:respawn:/usr/local/bin/monit -Ic /etc/monitrc

After you have modified init’s configuration file, you can run the following command to re-examine /etc/inittab and start monit:

telinit qFor systems without telinit:

kill -1 1

From the website:

Snare for Windows is a Windows NT, Windows 2000, Windows XP, and Windows 2003 compatible service that interacts with the underlying Windows Eventlog subsystem to facilitate remote, real-time transfer of event log information.

Event logs from the Security, Application and System logs, as well as the new DNS, File Replication Service, and Active Directory logs are supported. Log data is converted to text format, and delivered to a remote Snare Server, or to a remote Syslog server with configurable and dynamic facility and priority settings.

Snare is currently used by hundreds of thousands of individuals, and organisations worldwide. Snare for Windows is used by many large Financial, Insurance, Healthcare, Defence, AeroSpace, and Intelligence organisations to meet elements of local and federal security requirements, such as:

* ACSI 33
* GLBA (Gramm-Leach-Bliley Act)
* Sarbanes Oxley (SOX)
* C2 / CAPP
* DCID 6/3
* DIAM 50-4
* DDS-2600-5502-87 Chapter 4
* NISPOM Chapter 8
* HIPAA
* PCIDSS
* California Senate Bill 1386
* USA Patriot Act
* Danish Standard DS-484:2005
* British Standard BS7799

SNARE for Windows is free software (freeware), released under the terms of the GNU Public Licence (GPL).

Here are some screenshots. It seems to integrate with one of my favorite Windows-related sites (EventID.net), has cool filtering / alerting, and (perhaps the best part) is also open source! Here is an install guide, requirements, config tool, and the client agent.

Here is the description lifted their front page:

SB EventLog Monitor is tool for monitoring and consolidating Windows EventLogs.
Events can be collected: 1. using VBS script, that will collect all events remotly via WMI

2. using Windows agent, that will collect all local events and send them over HTTP(s) to the server.

After that, collected events are parsed using PHP and YOUR rules and are stored in MySQL database. There are two tables for this. All events are stored in the first one and in the second one only “Alerts” are saved. Alerts are Events, that matched against one of the rule. Alerts can be also send using email. Front end is web-based application with filtering and sorting.
Eventlog Monitor has 3 parts

Collector

First part is written in Microsoft VB script (executed using cscript) and used to collect EventLogs from computers using WMI (only new events are collected).
OR You can use the Windows agent for collecting events. Windows agent must be installed on each monitored machine. This agent will collect events and will send them over HTTP(s) to the server.

You can use both collecting methods together. ELM server can handle this.

Parser

The second part is written in PHP and it will parse all collected events and will store them in MySQL database. The “parse” also run user-defined rules on each event, so important events can be send using email or just stored (copy) in different table - so no important event is missed. Rules can also mark event as “noise” and then this event is hidden from standard views.

Web UI

The last part is web-based user interface for browsing the events and alerts. User can set filters, for better orientation. Web-UI is also used for configuring collector (which computers, how often, computer group); user control; rules creation and settings; cron control (how often are data parsed, when the old events can be deleted, …).

All is based on PHP(5.0+), web server (apache, IIS) and vbscript (and WMI). Only one scheduled task must be created, to execute collector and “cron” (PHP scripts for parsing data, sending emails, …). Collector must be run as user with administrative rights on monitoring computers!
VBS/WMI Collector

There are one little batch file and WMI script… The batch file will do web request (using curl.exe) to “prepare-bat.php” script and this is how, the “WMI-EVENTS.txt” file is created. Inside this file is just a list of target machines. Then WMI-EVENTS.vbs is called, and the Event collecting is started. ONLY NEW EVENTs are collected. All collected datas are saved as XML files. There are also “.dat” files - inside is the time of last event. When collecting is done, the “cron.php” is called (also using curl.exe) and events in XML files are parsed and stored in MySQL database. Thats the whole magic

The marketing mumbo jumbo on their website:

The Men & Mice Suite is a comprehensive DNS, DHCP and IP Address Management solution built to address the needs of medium to large enterprises. The Men & Mice Suite is the ideal solution for enterprises with Microsoft and Active Directory based networks, as well as those enterprises using Unix and Linux.

It is a challenge to manage a large IP address space, multiple DNS and DHCP servers, and all the users and administrators involved, using only the default management tools. The Men & Mice Suite meets this challenge and is the best tool for solving the scalability, manageability and troubleshooting issues in large network environments.

Deploying the Men & Mice Suite brings many benefits, among them the following three key benefits:

• Non-intrusive, step-wise deployment.
• Clear and integrated view of the whole network infrastructure, with the DNS, DHCP and IP address space merged into one unified updateable view.
• Easy task delegation enabling senior network administrator to safely offload daily network tasks so that they can be managed by local operators or help desk personnel in a secure and professional manner.

These (what I call) “meta”-infastructure tools are where the future’s at, in my not-so-humble opinion. Build out your network with these larger scale configuration tools in place, and you have a tremendous amount of flexibility (e.g. Puppet — which I’m starting to look more seriously at).

• Binary compatible (at least, in theory) with RHEL.
• Quite stable (the only time I get burned is when I run something like “yum update –enable CentOSplus (whatever)” on a CentOS 4 box)
• Yum packaging is now mature, and a company can easily make their own internal repository to cut down on update times.
• Updates for CentOS until 2010; til 2012 for CentOS 4; and (I believe) until 2014 for CentOS 5.
• Sensible file structures.
• Well put together packages that rarely step on each other (at least, in my experience).
• Solid enterprise support on enterprise level equipment (I assume because of the RHEL connection).
• Fairly good documentation.
• Certification paths (even though it’s through RHEL, it’s still very applicable).
• Different repositories for different purposes (e.g. RPMforge)
• In a way, it seems to be a “stable” version of RHEL.

Areas I see CentOS lacking in (but I don’t think really matter as much to lots of enterprises) include:

• Fewer packages (as opposed to, say, Debian or Gentoo).
• Yum not having as many “cool” options as other package management systems (e.g. Debian’s apt-get or Gentoo’s emerge)
• No minimal net install CD (like Debian’s netinstall.iso). The closet thing I have to that in CentOS is 4.x’s Single Server CD or popping in the first CentOS 5 cd and disabling everything (which is easy to miss if you simply take the CentOS 5 install defaults).
• Upgrading can be weird (What is the CentOS equivalent to Debian’s “apt-get dist-upgrade”?). I’ve experienced all sorts of hairiness when I upgrade PHP4 to PHP5 on CentOS, which I’m not sure is a result of PHP, CentOS, or my way of doing things…

On my personal time, I love challenges — funky triple boots “just because”, goofing around with xconf settings, lean and mean Gentoo installs on crappy hardware, or maybe sorting out problems that unstable packages bring. But when doing business, I want as few surprises as humanly possible.

For example, OTH’s radio archive doesn’t allow you to easily run wget (due to the pull down menus), so I used everyone’s favorite web scraper tool, Lynx, to grab the source, grep’d out the URLs in the pull down menus, sed’d these URL fragments into real URLs, and then piped these URLs in a file that was wget-friendly.

lynx -source http://www.2600.com/offthehook/archive_ra.html | grep /offthehook | sed ’s_”>.*__g’ | sed ’s_ __g’ | sed ’s_<optionvalue=”.._http://www.2600.com_g’ | sed ’s_<option selected value=”.._http://www.2600.com_g’ | sed ’s_<optionselectedvalue=”.._http://www.2600.com_g’ | sed ’s_\t__g’ > OTH

wget -r -l1 -t1 -nd -N -A.mp3 -erobots=off -i OTH

(Before I learned to use wget properly, I wasted time to using regex to wget files on their sites. That turned out to be a chore because their website names these files very differently. e.g. yyyymmdd, ddmmyy, yymmdd[a,b,c] (if it was a three part show, etc.)

I suppose my regex monkey work isn’t all wasted. I now have this big jumbled mess of file names that needs a coherent naming solutions, and regex seems to be the only way to go for that. (That’s a separate post once I iron out that script)
update: Here is a bash script by Peter Manis solving this same problem.

Since I’ll be using my company’s conference room, I’ll have a decent speaker phone. But since my company’s network and bandwidth is not really conducive to hosting stuff internally, I’m hoping to be able to stream out to some collocated server, and then everyone can connect to that server directly to see the presentation.

(Icecast + Theora look like a promising combo, but I have yet to implement either)

From the article:

Cfengine is a great way to scale common administrative practices — you can move from using SSH and a for loop to using Cfengine pretty smoothly. However, there is just as much complexity present in either form. You still have to handle file contents, and you still have to manage operating system differences yourself — you have to know whether it’s useradd or adduser, whether it’s init or Sun’s SMF, and what the format of the filesystem tab is.

One of Puppet’s primary innovations is a resource abstraction layer, so that you do not have to know those details. You can speak in terms of resources like users, services, or filesystems, and Puppet will translate them to the appropriate commands on each system. Puppet administrators are free to focus on the complexity of their configurations, rather than being forced to also handle that complexity plus the complexity of the differences between the operating systems.

Puppet’s development was heavily influenced by the many external modules that Luke Kanies wrote for cfengine, each module managing a separate resource like users, packages, or cron jobs, and one of Puppet’s primary goals was to be able to make it easy to expand the number of resource types it can manage.

From the page:

SNMP Tutorial Part 1: An Introduction to SNMP

The first part of our SNMP tutorial series introduces SNMP with a brief history of the procotol. In this lesson, you will learn about SNMP network architectures. You will also learn about how the various parts of an SNMP system interact to convey SNMP traps to your SNMP system master. This includes different types of common SNMP commands that you can use within your own SNMP management system.

SNMP Tutorial Part 2: The SNMP Management Information Base (MIB)

The first part of our SNMP tutorial series introduces SNMP with a brief history of the procotol. In this lesson, you will learn about SNMP network architectures. You will also learn about how the various parts of an SNMP system interact to convey SNMP traps to your SNMP system master. This includes different types of common SNMP commands that you can use within your own SNMP management system.

SNMP Tutorial Part 3: Understanding SNMP Packet Types and Structure

This module will teach you about the structure of SNMP trap packets. This includes a review of the types of SNMP command-response packets for common SNMP traps. You will also learn about the formatting of SNMP trap packets within the SNMP MIB.

SNMP Tutorial Part 4: Layered Communication

The fourth article in the SNMP tutorial reviews the layered communication model of SNMP. This includes a description of the various communications layers, and what SNMP applications take place at each. This will also explore the assembly of SNMP packets within the Network Interface layer. Helpful SNMP layer diagrams will teach you how to troubleshoot problems by understanding layered SNMP communications.

SNMP Tutorial Part 5: Common Mistakes Made When Integrating SNMP and Non-SNMP Systems

The final module in our SNMP tutorial goes over common mistakes made when integrating SNMP and non-SNMP systems. This module is particularly important for network operators and engineers when deploying or further developing any type SNMP communications system. It will teach you seven common mistakes operators make when buying SNMP equipment, and the requirements you should look for when purchasing an SNMP monitoring system.