Skip to main content

The Tectonic Speed of Government, Part 1: Procurement

This post is my reaction to conversations about how hard it is to create change in government, and how government projects (and IT projects in particular) take so long from genesis to completion. This is part 1, about procurement; part 2 will address project implementations. PS - there was a surprise Part 3 of this series later!

Warning: what follows is an “inside baseball” discussion of government IT procurement. I’m not trying to dissuade you from reading it, but if you’re not enmeshed in this world you might want to consider reading my articles on lighter topics like organizing your electronic life, the greatness of Abbey Road, or the story behind The Room.

If you ARE enmeshed in this topic, then please don’t overlook my call to action at the end! Changing the process will take a group effort, and I’m hoping to get feedback on my scheme to create a library of reusable software specifications.

By the way, this post’s first title was “The Glacial Speed of Government” but that title is both cliché and inaccurate, as today it implies a faster pace than it used to. I decided that “Tectonic Speed” is more accurate, as change in government happens slowly but with occasional significant outbursts. It is my hope that one of those is coming soon, so let’s do what we can to bring it about.


Disclaimer: All of the ideas expressed in this article are my personal statements and opinions, and do not reflect the opinions/statements of the City of Urbana.


The Tectonic Speed of Government, Part 1: Procurement

The suckiest part of my job (as a City IT Director) is buying things. Actually, it would be more accurate to say “buying non-things” – because it’s relatively easy to buy physical STUFF, like a printer. What’s hard is buying intangible IT purchases like consulting, training, and software.

The problem is that government purchasing is built around buying STUFF, not non-things, because historically that’s what governments bought. (One exception: Public Works has a well-defined process for engineering consultants.) Even early IT purchasing was still STUFF like computers, tape drives, and software that you acquired once. Now IT purchases are so conceptually vague that they’re reduced to catchy idioms like “Software as a Service” (SaaS - a fancy phrase for renting software) or “Infrastructure as a Service” (IaaS – running components of your network on someone else's machines). Procuring these isn’t like getting road salt, or even printers.

Government procurement in brief

Let’s pause here for a little background. In most cases, public sector purchasing has three levels of effort, based on the cost of the item.

1. For low-cost items (less than $2,500… I’m using our City’s policies approved in Feb 2018, which are as close to a “market rate” as you might get! - NOTE #1) the person who oversees the budget has leeway to buy whatever they want as long as they can justify the choice to whomever chooses to review the purchase.

2. There’s a middle range (above $2,500 and up to $10,000) where if we need STUFF like a printer, we identify the specifications we want (“color laser printer that produces 30 duplex pages per minute”), then we request prices from vendors that meet those specs. We need to get at least three comparable prices. Usually, we buy the lowest cost option, although we can choose an alternate option if we have good justification. For this range, additional approvals may be required before the purchase is consummated.

3. The highest level ($10,000 and up), requires a more complex process in which we create a document called a “Request for Proposal” (RFP) or “Request for Bid” (RFB). (I’ll refer only to an RFP from now on, but I mean both.) We prepare the RFP to explain what we want and tell Vendors up front how the winner will be selected - usually a mix of price and solution. Vendors respond with Proposals. Depending on the purchase, there might even be a committee reviewing the proposals and selecting a vendor.

The problem in the status quo

The process sounds reasonable, right? But imagine a government process that was built as a reaction to spending scandals with over-paid awards to buddies of influential people who sold shoddy products. Well, the pendulum has swung and there's now a substantial focus on objective criteria and chronicling decisions. I'm OK with that, and I believe that the process to buy items in the first two tiers is appropriate as long as the chronicling is simple, effective, and transparent.

The problem is buying in the most expensive tier. Sometimes the RFP requires more work than is justified by its benefits. And I say that with the assumption that the RFP is done efficiently, where the majority of the document is boilerplate and you can focus on the part that describes what you want. (NOTE #2) The part that describes what you want must be explicit, so it can be used to compare proposals as objectively as possible. Usually, these are presented as detailed “specifications”.

The use of specifications is a direct result of the historic purchasing process, in which you submitted a purchasing request to a Buyer whose job it was to find a suitable item and purchase it. Buyers became specialists in certain goods (e.g. office supplies or vehicles), or at least had expertise in conducting the process. But when Buyers didn’t know the items well, they depended on good specifications.

Unfortunately, the past few decades' explosion of technology coexisted with another trend in government: reducing administrative staff. One casualty of that were Buyers, because in a world with Amazon, why do you need someone whose specialty is the office supply catalog?

Larger organizations may still have a Purchasing Department staffed with Buyers, but for smaller cities like Urbana those days are long past. Middle-level managers are now responsible for pricing and buying ourselves, even though the process remains just as complicated as before. Writing specifications and chronicling decisions are time-consuming steps we must follow, especially when you’re asking to spend $100,000 on cloud-based software that you’re selecting based on the “User Experience”.

Which brings us back to the core problem: how do you apply a 20th century process designed to buy STUFF to our current needs for non-things?

As I’ve written about elsewhere, writing specifications for non-things is a difficult process that can only approximate the ideal. (Briefly: how do you write a series of declarative sentences that completely define your non-thing needs? Answer: you can’t.) It’s worse in IT, because any technology we buy is new – and how do you describe what you want when you’ve never used it? And we can’t just buy the cheapest one, because each of them has different capabilities, and none of them do everything we want. You’ve dealt with a similar conundrum if you’ve shopped for smartphones and had to consider trade-offs in screen size, memory, and price. (Unless you simply buy each new Apple product when it comes out, in which case I don’t recommend a career as a municipal IT professional!)

Ultimately, the RFP is more of an art than a science, and always requires subjective decisions, which can be contested by unsuccessful vendors, should they choose to. So, we must contort a subjective process and force it into objective criteria for decisions. (“Hmmm… I’ll score it an 83 on User Experience.”)

Since the RFP process won’t change quickly, here are a few suggestions for those who must live with it. 

#1 Bypass the process when you permissibly can (and it makes sense to do so)

There are some shortcuts through the complicated process. A popular option these days is the “joint purchasing” cooperative, where one entity conducts a competitive procurement - so they do the hard work once - and then all partners can purchase from the winner(s) without any further extensive process.

This isn’t new: local governments buy from State purchasing contracts whenever we can, but it’s a limited set of items. (Urbana IT uses use State contracts for printer ink refills and mobile phone service). When buying from a State contract, I am allowed to bypass all of the pricing and RFP steps, because the State has already done that for us.

What’s different with the new cooperatives is the breadth of contracts available, even including… non-things! You can purchase software at a fixed discount, for example.

I’ve seen joint purchasing work well already: we needed copiers, and were able to ditch a complicated RFP (after I’d *grrrr* spent many hours writing it). Instead, we chose two vendors through the NJPA purchasing cooperative, then asked them to price the same scenario of lease + service, based on historical copier usage.

It worked wonderfully, but buying copiers isn’t truly a non-thing. With copiers, you have a clear idea of what you’re buying, and a fixed-price service contract to support it. Plus, you can compare copiers on criteria like pages per minute and tri-folding.

Now I’m experiencing my first attempt at buying software from a joint purchasing agreement, and I’m realizing with great disappointment that this doesn’t seem to be a good match.

The problem with joint purchasing is also its best feature: cooperative members can bypass the RFP process. (Let’s be honest here.) If it’s easier to buy from the pre-screened vendors then we'll do that, because that’s human nature.

One impact of that is a network effect that rewards the small number of vendors who are selected by the cooperative. Ultimately, this is a loss for consumers, because a larger pool of vendors creates more competitive products, and lower prices.

With non-things, bypassing the RFP means skipping the RFP’s one great benefit: writing specifications forces us to think about and describe what we want. Regardless of how poorly we misjudge it, the very act of writing it down makes project success more likely. (“Plans are useless, but planning is indispensable.” – Dwight Eisenhower.)

Writing the RFP also improves our cost estimates. An analogy: implementing a software solution is like hiring someone to build your house, and writing an RFP is like drawing a picture of your ideal home’s floorplan first - before you get cost estimates from builders. It gives them an understanding of what you want, and an opportunity for them to say, “you know, you could save a lot of money by not putting the garage upstairs.”

Continuing the analogy: buying from a joint purchasing agreement is like selecting a high-rise building, then choosing from the floor plans. Odds are you’ll see something you like, but customizations will add costs, especially if you don’t consider all your needs at the start of the process.

Joint purchasing lets us be lazy and skip this needs assessment... and it's really tempting to follow that easy path. “Surely,” we justify to ourselves, “this vendor’s software must be good – after all, they were selected!” But you don’t know whose specifications the cooperative used to choose the vendor, or if there were any specifications at all.

So joint purchasing agreements are a valuable tool, but in their current state you’re trading off the ease of buying against finding a good fit and price. A good strategy therefore is to do the needs assessment first (the high-value part of the RFP) and then buy from a catalog with a clear view of your prioritized needs / desires / wishes, etc.

#2 Recognize that even non-things can be bought like STUFF

Municipal government is chronically late to the technology party – so the non-things we buy may be commoditized by the time we purchase them. The Internet “Cloud” is a great example: dozens of vendors are ready to sell you cloud storage, cloud e-mail, etc. It would have been much more difficult 10 years ago.

My proposal here is that anytime you are able to get apples-to-apples pricing, getting three comparable quotes should be sufficient. This requires rethinking the dollar limits; these are high-dollar purchases, due to the technology involved, but in reality it’s a simple price comparison scenario.

For example, backing up data to the cloud. It would cost more than $10,000/year to store 40 TB of files on the cloud. That would require an RFP, but ultimately we’re just pricing the same commodity (storage costs per Terabyte) from Amazon Web Services, Microsoft Azure, and other cloud providers.

Here’s a horror story: Recently the maintenance agreement on our IBM mini-mainframe was up for its triennial renewal. Multiple resellers all offer the identical IBM maintenance agreement. Due to the dollar amount of a new 3-year agreement, I had to write an RFP. Due to confusion about our license in the IBM records (don’t get me started on that topic…) I spent at least 40 hours documenting our needs in an RFP, sending them to resellers, answering questions, and analyzing the proposals. Add in the costs from Legal and Finance and we spent thousands of dollars in salary costs on the RFP. Ultimately, the winning vendor had the best price by .206%. ($44.40 out of almost $21,500.) It pains me that I could have gotten the same bids just by sending out three e-mails. (NOTE #3)

More than the salary cost, it’s the opportunity cost. 40 hours is more than 2% of my year! What other tasks and projects could I have accomplished with that time? 

#3 Improve the purchasing process through technology

I’m not even going to delve into using technology to manage the flow of procurement within the organization. Bottom line: information should be easy to enter and automatically carried forward across every step in an auditable way. Full stop.

Let’s assume we’re not getting rid of the RFP at this point, because there is value in defining what you want. And let’s assume that you’re working in an organization that has an efficient RFP process, with boilerplate text to use for the RFP and a blank portion where you insert your specifications.

So, how can we use technology to improve the task of defining specifications?

My proposal: a crowd-sourced specification database. It would work like this:

1) We start with an initial set of specifications that someone wrote. (Then, we ask them to revisit the list after the project and modify them based on “what we wish we had asked for!”)

2) We post that list to a structured database, where they are categorized by business function and use case. For example, in the business function "Police Evidence Intake" Use Cases would include "Record Evidence," "Evidence Lab Testing," and "Evidence Inventory." Specifications would then detail those processes: "Record an unlimited number of lab test results as child records for a single piece of evidence."

3) People who need to write an RFP would use a website to select business functions & use cases that they want to address, then they could review the associated specifications and select the ones they like. If they don’t find appropriate specifications, they can suggest changes or add their own within the same structure. (Using a Wiki-type tool, the user community could peer-review suggested changes.)

4) The underlying database will track how frequently specifications are used, with more frequently used ones bubbling up to the top of the list, and unused ones periodically culled. (Chaos Theory applies to data: you need constantly organize, or else it devolves into a complete mess.)

I propose that specifications for municipal software are a great place to start. (Although it’s not limited to that, small business owners could particularly benefit!) I’m starting with municipal software for several reasons:

  • Municipal needs are fairly standardized. Urbana's needs are broadly the same as Peoria, Ketchikan, and Los Angeles because we do many of the same functions.
  • Municipalities have a wonderful share-first mentality. We happily borrow and lend all kinds of IP to each other.
  • Defining software specifications is difficult, especially for software that we don’t have. We lack expertise and may not have the budget to pay for help.

There’s a later step in this vision, in which vendors could review commonly used specifications and prepare answers, which could be loaded to the database also. (NOTE #4) This would speed up the vendor’s process for responding to these items, also, and maybe even allow people to preview how vendors address those needs before the process begins.

For vendors, there could be some additional benefits. There could be some standardized EEO information captured, avoiding their need to complete different paperwork for every entity. On a similar vein, they could record basic information like W-9 forms, references, and contact information in one place for reuse.

I did some research to see if I’m reinventing the wheel, but I couldn’t find any direct equivalent with a database of available specifications to use. There are companies who offer government procurement software, but they're focused more on supporting the internal process. One of them described a library of RFP text, but none of them mentioned standardized specifications for easy reuse by anyone. (This makes sense: they’re for-profit companies, so they have an incentive to have a “gated” community approach.)

So, that's my proposal. I do acknowledge that as a systems analyst who has spent a career designing data structures, my proposal has a bit of “if all you have is a hammer, everything looks like a nail.” Still, I feel like a centralized database of specifications is the right tool for the job. (Another tool-based quote: “If it's for digging a hole it should probably look something like a shovel.” – John Gall in "Systemantics")

At this point, I welcome a dialogue on this proposal, and how to move it forward. 


Note #1: Until February 2018, Urbana had ridiculously low dollar thresholds. The first limit was $1,000 and the second limit was $5,000. These dated back many years, and I'm thankful to our Finance Director and Purchasing Committee for rationalizing them.

Urbana's low dollar limits succeeded in the intended consequence of impeding spending, by making it a hassle to buy things. And they had the unintended consequence that the City didn’t invest in technology or modernize its systems. As I’ve discussed elsewhere, even now in February 2018, we are still using homegrown green-screen applications for the majority of our business systems. Good news: we transitioned from one system in 2017, and we’re are leaving at least two more in 2018.

Note #2: If done inefficiently, the RFP is a painful exercise that saps many hours of your life. (Bad memories… let’s not go there.)

Note #3: Actually, we could probably have lowered our costs by skipping the RFP because some vendors don’t waste their time writing proposals, and the ones who do that extra work bake the costs into their pricing.

Note #4: The database would track if there are changes to the specifications that post-date the vendor answers, of course, so Vendors could see changes to the wording of the specifications and adjust. It needs to be efficient for Vendors, too!

Popular posts from this blog

The Tectonic Speed of Government, Part 4: Momentum This “Tectonic Speed” series is about why Government IT Projects take such a long time. The name refers to tectonic plates, rubbing against each other. No visible movement for a while then… CRACK! Government change is like that; it can take a long time to build, but when it happens it can be intense.  For more on that here is my 20:50 speech on this theme from the Code for America Summit 2020 , which turned into a virtual event. (I can tell you that it’s 20:50 because of the PechaKucha-ish format: 25 slides for 50 seconds each.) ************************************************************************** Disclaimer: All of the ideas expressed in this article are my personal statements and opinions, and do not reflect the opinions/statements of the City of Urbana. ************************************************************************** One impact of the COVID-19 pandemic is a shake-up of p

How To Videos: Lucity Queries with Microsoft SQL Server and Excel

What follows is not a blog, but some suggestions on using Microsoft SQL Server "Views" to query your Lucity data using Excel.   This information is intended to assist Lucity software users, and not for any nefarious purposes. I recommend watching the videos in Full Screen view and with HD resolution.  They're not as blurry as they look on this page!!  Each of these about two minutes long, but the original actions only took 50 seconds each.  (After recording them, I decided to slow them down to make them more watchable.) 1. How to create a SQL Server "View".   The video shows how to create a new View from the core Work Order table.  (WKORDER - see the data dictionary here .)  The video first shows the simple method of creating a view with all fields, then shows the more effective method of including only needed fields, and re-labeling them with their on-screen names. Music: "A View to a Kill" - Duran Duran 2. How to Connect to the Vi

Why the Tectonic Speed of Government?

The original name ("Glacial Speed of Government”) is both cliché and inaccurate, as it implies a faster pace than it used to. I decided that “Tectonic Speed” is more accurate because change in government shows tremendous resistance and moves slowly, but when it happens progress can occur in significant outbursts - and in those moments, there is great opportunity!     Click on "Read the Whole Thing" to access these links: Part 1 |  Part 2   |  Part 3  | Part 4 | Wait there's a Video?!  

But First... Let Me Take a Selfie

Excel 3D Map of my drives home. Colors are waypoints and height is the trip duration. Read on to see what the hell this is. This piece began before the pandemic, but the use of our phones as tracking devices for contact tracing prompts tough questions about privacy and tracking that make this timelier than I expected. See the middle section for thoughts on this. *********************************************************************************************** Disclaimer: All of the ideas expressed in this article are my personal statements and opinions, and do not reflect the opinions/statements of the City of Urbana. *********************************************************************************************** “OK Google… Track Me All Day Long”  Google Location History logs your every move, every day – if you turn it on. (To their credit, it’s off by default.) It’s worth asking - why would anyone want to do this?  Well, I did it for a few years because I was curious about the r