Wednesday, October 14, 2009

Liveblogging p&p summit, day 3, designing for performance

Yes, let's all agree that all apps must have a performance plan.
  • Highest-priority scenarios - could be response times to user input, or server throughput, or other. I'm thinking data response times too.
  • Articulate Bad, Good and Excellent performance - e.g., on startup 10 sec is bad, 3 sec is good, <>
  • Do coarse estimation if "good performance" is in jeopardy - if uncertain, prototype and measure more; if bad, redesign.
Design design design design. "If you have a performance catastrophe - 10x, 100x from acceptable - then you either have some serious bugs or the flaw is in design. If it's in the design, then you're screwed. You cannot fix this late in the product cycle. You have to start over and redesign the app from scratch for version 2." You don't say. J.f.C.

Measure. Use real numbers. Measure early. Don't skimp on this. Performance is not free.

You cannot commit to features or designs until you know how much they cost, and to know that, you need real facts and real numbers. Use references (consults?), past experience, and experiments/prototyping to determine what things cost. You can only make rational decisions about how to build things if you know this. (I am sure this must apply to aspects of decision-making beyond just performance.)

Multi-threading. Ugh.

A symptom of blocking - you have a ton of stuff going on/waiting, but the machine's CPU is not fully utilized. This means something's hanging on some thread while the machine has idle threads to spare. Points of communication between threads will then be the bottlenecks. It's fine to share RO data between threads because each thread can keep its own hot copy.

Disk is 10,000x slower than RAM.

"The problem with caches is they only work the second time." D'oh.


Memory is not a primary metric - time is. It only matters inasmuch as it affects time.


1. Care.
2. Plan.
3. Measure.
4. Design.
5. Measure some more.

Tuesday, October 13, 2009

Liveblogging p&p summit, day 2, keynote

"Don't allow an unbelievable schedule to live."

Not just don't, but here's why not:

When people don't believe the schedule, it undercuts their belief in anything else you say.

When everybody "knows" you can't make the schedule, if you're not actively managing that, they'll start making their own decisions and judgments. If this is under the table, then they won't do it the way you (the manager) would want. E.g., if they "know" the ship date isn't realistic and no one is talking about that, maybe they'll speculate about how much it's going to slip, and start working to that assumption. If you (the manager) don't know what they're speculating and aren't giving realistic, believable input into that judgment, then you won't have any influence over that decision they're silently making about how to work. Yowch.

The usual solutions are to change the ship date or change the scope, to bring about a believable schedule. Both of which are the right things. I suppose there is another possibility, to convince people that the original scope & schedule should be believable, by providing lots of supporting facts and Gantt charts and suchlike. The problem here is that this is more like browbeating, most of the time. If your developers don't believe the schedule (assuming your developers are minimally competent and are at least somewhat able to explain their concerns rationally; if not, then you have a much bigger problem), chances are you should listen to them instead of trying to talk them out of it. As well, not listening to them makes all of the above outcomes worse.

"Working harder is a short-term solution." Death marches R bad, mmmkay?

Monday, October 12, 2009

Liveblogging p&p summit, day 1, more

Gratified to see that my team isn't the only one with an unhealthy fondness for layer cake metaphors (and Visios)!

"Customer doesn't care whose fault it is; they just want things to work." Good reminder for me, personally.

Got to talk to a very delightful p&p program manager/session presenter in the coffee line while a different session was going on. If I were actually any good at this "networking" thing, I would have figured out a way to snag a business card or thought of a thought-provoking question for further follow-up. Live & learn.

"When [users/developers] get a new version and have to spend all their up-front time fixing compatibility issues, they tend to not want to use your app at all." Not that Microsoft would know, of course. ;)

Much later in the day...

Experimental 15-minute "lightning sessions" debuting this year. The first one was a Program Manager discussing transition to Agile, which was disappointing because it didn't cover Agile in very much depth. In 15 minutes. I know! Really! (Ahem, kidding.)

The current "lightning session" covers a concept called Behavior-Driven Design. I love it. This is either something we are already accidentally doing, or something I wish we were doing. Either way, it seems to me that it would fit well within the methodologies we already use. Awesome.

I exercised restraint and did not "mention" the speaker, @ElegantCoder, in a tweet, even after other attendees did and his TweetDeck notifications started popping up on his presentation laptop. The temptation was tremendous.

So he uses the word "scenario" for the "given/when/then" construct describing a desired/expected behavior. We also use the word "scenario" and I'm trying to figure out whether our usage is reasonably compatible with this standard definition, or if we're abusing it. We use it to describe automatedly-testable behaviors, so it does seem to fit together, but it doesn't really describe our internal architecture, so maybe not so much.

OK, this is cool. Third lightning session on SharePoint development seemed ridiculously irrelevant to our team, but instead, here he is demoing the concept of Inversion of Control, which is perfect for some of the team members we brought along, and the examples are simple, and he's explaining the value IoC adds to the process. Hooray!

That means 2.5 out of 3 lightnings are useful so far.

Last lightning round: Billy Hollis. Always a total, major win even if he doesn't fully appreciate that there may be female coders in his audience (or in the universe) when he writes his jokes. Ahem. *dusts shoulder-chip* I'll still give him full credit for "useful". 3.5 out of four, nice work p&p.

Liveblogging p&p summit, day 1, intro & keynote

ZOMG it's Martin Fowler. Right now! I knew he'd be here, but, first! #fangrrrl

"We need to reduce our emphasis on quality so we can fit more features into the next release"? No, that's definitely never happened to me. Nope.

Attendance is way, way down compared to two years ago. Our own delegation this year is half the size (or smaller?) than ours then, and I'd really guess the overall crowd size is similarly proportioned. Which makes me wonder about last year (immediately mid-meltdown) when we couldn't attend at all. Don't know if it is valid to think of us as a bellwether.

Lord, this clacky Dell keyboard is already not making me any friends here.

Key concepts from the Martin Fowler leadoff keynote, part 1:

External quality, which is negotiable with the users/product owners (e.g., they get to prioritize bugs vs. new features), vs. internal quality (architecture), which really mustn't be.

Accidental complexity vs. essential complexity. Code debt!! Initial accidental complexity is the principal and working with the accidental complexity when implementing new features/changes is the interest. This impacts the effort required to do that new work. It isn't just "debt", it could be on the scale of a technical subprime mortgage. Not that I would know. Nope.

Debt planning (refer also to the debt payoff line graph):
Deliberate/prudent: "We must ship now and deal with the consequences"
Deliberate/reckless: "We don't have time for design"
Inadvertent/prudent: "Now we know how we should have done it"
Inadvertent/reckless: "What's layering?"

Event sourcing:
(This is relevant to my interests.)

We log all events in an insert-only store, and we trust our log, such that at any time we could rebuild the entire current application state (e.g., dB) by parsing the log. This also means we have a solid audit trail, and we could rebuild the entire historical state of the application (presumably someplace else) just by dialing back the date. Or diff two states. In other words, this stuff isn't just for version control systems any more.

Also useful for various other things which hopefully one of my colleagues took notes about. Oops. #shortattentionspan

OK, now there's a dude in front of me taking pictures of Martin Fowler. Unless he's part of Microsoft's PR division, that means I am not the only #fangrrrl in the room! I am sorely tempted to take a picture of him taking a picture.

Coffee time!

Monday, August 24, 2009

Algebra in real life, part II

This post is dedicated to the memory of Mr. Earl Wog, who taught us about Joe Pythagoras and also taught me how to drive. :)

I've needed to know this two different times in the last six months or so, and had to figure it out from scratch both times, so it's time to document it for posterity.

When you're trying to get payment from somebody via, say, an online credit card processor such as PayPal, and the processor takes a percentage of the total transaction as its transaction fee, how much do you need to charge in order to receive the amount of money you actually want? Here is the correct formula:

[amount you want to receive] ÷ ( 1 - [fee, as a decimal] )

The following, seemingly plausible formula does not work, and will result in you receiving less money than you wanted:

[amount you want to receive] + ( [amount you want to receive] × [fee, as a decimal] )

Pro tip! If you accidentally used the second formula instead of the first one, don't make an indignant post in the credit card processor's user support forums demanding to know why they're shorting you.

Thanks, Mr. Wog, wherever you are!

Update: I received a suggestion that the following formula may also yield the desired result:

[amount you want to receive] × ( 1 + [fee, as a decimal] )


x(1+y) == (x × 1) + (x × y) == x + xy

which is the same as the second formula above and isn't correct. Thanks to GAM for helping me with the proof on this. Make no mistake, my algebra skillz are tenuous and that's why I have to struggle through formulas and post them on my blog. :)

Saturday, December 20, 2008

WTF is it with Seattle and snow?

I survived four years of Massachusetts winters in college. That's mostly where I learned to drive; I picked up some snow skillz during shifts piloting the (16-passenger, RWD, usually-empty) campus shuttle van. It just wasn't that scary.

One year I invited a Mass-native friend (hi Katy!) home to see Seattle during winter break, and my family wouldn't let us go anywhere after < 1" of snow fell on the city. She was disgusted; I was mortified.

So what is the deal with snow in Seattle? If, e.g., Bostonians, had our attitude about it, they'd be forced into hibernation six months of the year. Non-natives never seem to tire of pointing out what pantywaists we Seattleites are about this.

Most obvious reason: It rarely snows in Seattle. My entire childhood, twice a year (November and March), lasting 1-3 days, if we kids were lucky. This leads to side-effects, and not just the one where Seattleites hardly ever get to practice driving in snow.

Seattle and King County own fewer snowplows than other cities. Many or most snowplows are fitted with rubber blades to keep from damaging the extensive network of raised-button and reflective lane markers:

Roads crews are working around the clock to continuously plow and sand City streets.... Following the plows are the sanders to provide traction on the ice. Snow plows’ rubber blades do not remove ice.

(We use raised lane markers because it rains here. Ever seen the lane markings when it's wet in Boston? Neither have Bostonians.)

No wonder most of Burien today looked more like it had been polished than plowed, though.

Seattle and King County say it is not cost-effective to maintain any larger a fleet of snowplows or sanding trucks for how infrequently they are needed. This seems reasonable to me, unless these last few years of storms are harbingers of long-term climate change or something.

On the flip side of this, as alluded-to above, cities with regular heavy snowfall get good at dealing with it out of practical and economic necessity. People couldn't live there if they didn't. People would have to move to friendlier climes, like Seattle... hey, waitaminnit....

A related reason: Snow in Seattle is almost always wet snow. Even when it gets cold enough for precipitation to fall as snow, we're usually flirting with 32°F.
Several of you commented about the nature of the snow last night. Most of you are used to the large, dendritic crystals that fall when temperatures are near freezing...our usual situation. Last night you got to enjoy the type of snow they get in colder climates.
Most of the time, that means what little snow we got will melt away quickly, completely. But when we get snow of any quantity, often, it'll melt partially during the day and re-freeze as sheets of solid ice.

Also, wet snow compacts differently when driven on than dry snow does, which is especially relevant when the streets aren't getting plowed right away (or at all).

A frequent complaint: We don't salt the roads. The poor salmon! Think of the salmon! (Or is it "think of the undercarriage"?)

Another obvious, though debatable, reason: Seattle is really, really hilly. Stuff other places call "mountains", we call "housing developments" and "arterials". I have to negotiate several steep hills to get out of my neighborhood in any direction. Only one of these is ever sanded; none are ever plowed. It is claimed, however, that other actual hilly cities manage better than Seattle does. See above and below.

The oft-cited reason: "I can drive fine on snow... it's all those other maniacs." Does this refer to all those native Seattleites who can't drive on snow? The natives I grew up with refuse to leave the house at the first sight of flurries. Could it be all those transplanted drivers zipping around assuming our roads are as driveable as the ones where they came from? (Maybe it's just the free lobotomy given to both kinds of drivers when they buy an SUV.)

Case study: This super-awesome news story from yesterday, wherein two charter buses nearly plunged 20 feet onto I-5 after trying to take an icy hill without chains, arguably had several of the above causes:
  • Steep urban hills
  • Closure of an unplowed arterial
  • Icy Side Street of Death™
  • Some kind of driver cluelessness, or reckless bravado, which led the first bus to ignore pedestrians frantically trying to wave it off its ill-fated left turn and the second bus to make the same turn after the first bus was already sliding and the second bus' passengers were screaming at the driver to stop
What have we learned? I dunno, but Washington state seems to do reasonably kind of OK with that big ol' mountain pass we have (I-90, known elsewhere as the Mass Pike); we manage to keep it open most of the time, even during avalanche season, so somebody somewhere in this state must know something about making roads driveable in snow. Maybe just not so much down here at sea level.

In conclusion: A couple years ago, I waited a few hours after the snow had started to begin my trek through the city from work toward home. I Had a Bad Experience. Two to three HOURS of road closures, crazy heavy traffic on unplowed urban side street detours, unwise steep hill attempts blocked by other people's earlier unsuccessful unwise hill attempts, jackknifed Metro buses, and finally utterly fucking clueless pedestrian neighbors who let their kids and dogs frolic in front of me on the steep unplowed hill I was, at that moment, sliding down uncontrollably. ¡No más! I got nothin' to prove any more, and I ain't goin' out in this stuff if I don't have to.

Bonus update: Dear Science explains how your SUV is subject to the same Newtonian physics (hi lafe!) as the rest of us, no matter what the dealer may have told you.

Another bonus update: Cliff Mass wonders whether it's really more cost-effective for Seattle to skimp on plows:
It is true that having extra plows for city trucks are not free and that snow events like this are unusual. But the economic loss of allowing the city to be crippled by such modest snows is substantial...and major decisions (like the cancellation of schools last Wednesday when no snow fell) are made in the context of such poor snow removal.
It would be interesting to try to quantify, indeed.

Friday, November 14, 2008

Bold, bold choices

Last night on Countdown, I swear* Keith Olbermann said something about how President-elect Obama's hypothetically possibly choosing Hillary Clinton as his Secretary of State would be a bold move because it would place a woman in such a high office.

A few moments later he clued in and noted that the current Secretary of State is Condoleezza Rice.

Great job, Keith, except Madeleine Albright was appointed Secretary of State by President Bill Clinton in 1996 and served through 2000.

[insert "Worst Persons" ominous theme music here]

But wait! There's more!

This morning on KIRO AM 710, Dave Ross said something about how President-elect Obama's even-more-hypothetically possibly choosing Colin Powell as his Secretary of State would be a bold move because it would place an African-American in such a high office.

He did not manage to clue in that the current Secretary of State is Condoleezza Rice.

And, somehow, even though this was the whole point of the discussion, did not put it together that Colin Powell, having been appointed in 2001 by President GW Bush and having served through 2005, has already been Secretary of State.

In conclusion, people:
  • We have already had two female Secretaries of State.
  • We have already had two African-American Secretaries of State.
I realize that we are really, really enjoying the historic-ness of the present moment, but let's come to grips with the fact that the State Department's historic moment has passed.

Hell, I'm not sure even an openly gay or lesbian Secretary of State would be that bold a move... unless she or he has the temerity to want to get married or something.

(Seattle March for Equality tomorrow, November 15, Volunteer Park, 10:30 AM with keynote speakers starting at noon...!!)

* I did not rewind or write down the exact quote or look online for the video today, so standard disclaimers apply.

Tuesday, November 04, 2008

Election Day transcendence

I tempted, no, taunted fate by wearing a RED shirt* to work on Election Day.

I was pleasantly surprised to find that Kyle wore a blue one. (Certainly without as much "meta" or superstition as me.)

I would like it very much if we could follow President-elect Obama's lead and be less "red" and "blue" in our collective political thinking. This is not to say that I'm turning away in the slightest from my values, nor Kyle from his, which will make it an interesting exercise.
* Not a Star Trek reference.

Election Day synchronicity

Interesting mix here in my Twitter feed.

Now go vote!