posted by Tom Moertel at 11:31 AM /
Update 2011-06-15 15:00: add more links to media coverage.
Update: another briefing on the high-school project is scheduled for next week at 6:30 pm, before the regular school-board meeting on Monday, June 20, 2011. The meetings will be held in the high-school library.
Before last night’s school-board meeting, there was another update from the architect and construction manager on the effort to reduce the price of the high-school project. The meeting was scheduled to run for 30 minutes; it ran for about two and a half hours.
There are too many details to recount; if you want them, listen to the meeting recording over at Lebo Citizens. But here are some interesting things I observed.
First, there is evidence of progress: a list of about 200 options for reducing the project’s cost. I think I heard it said that the list represents a potential $14-million savings. That’s the good news.
The bad news is that there is evidence that something’s not adding up. School director Dan Remely noted that the project’s cost per square foot is still unusually high. Even with the proposed savings, we’re only saving a couple of dollars per square foot, he noted.
The project was originally estimated at about $185/sq-ft but the actual bids put it at about $220/sq-ft. This graphic, which I prepared in January, 2010, shows how unusually high $220/sq-ft actually is. In short, 90% of similar projects from 2008 cost less than $150/sq-ft. Our project costs that and nearly half again more.
Further, School director Ed Kubit shared some of his back-of-the-envelope calculations and said that, if the proposed cost reductions were accepted and had their intended effect, their savings would just reduce the project’s cost to the previously declared maximum of $113 million – if applied to the lowest bid we got in April. He pointed out, however, that we might not see bids that competitive in the next round of bidding. The margin for error was slim.
One thing that concerned me was that head architect Tom Celli “urged” the school board to consider switching from single to multiple prime contracts. The general understanding is that this switch would reduce the construction bids – making it more likely that a bid or two would squeak under the maximum – but, ultimately, cost the community more during actual construction. Most school directors, thankfully, seemed to be skeptical if not resistant to this advice.
Another thing that concerned me is that it wasn’t clear what effects all these proposed changes would have. School director Lawrence Lebowitz said that he was concerned about “what is coming out” of the project. Other directors echoed his concern, saying that they didn’t even know what the exterior of the building was going to look like any more. The architects said they could furnish “elevations” to help the school directors visualize the changes, but a 3D computer rendering would take too long to prepare.
If you’re getting the picture that nobody seems to have a handle on these changes, you’re seeing what I’m seeing. Some directors see it, too. I heard at least two say that the timeline for considering changes was too fast. They felt rushed and were concerned that, if they had a hard time keeping up, the public was probably even further behind. Two directors said that there ought to be a public forum to make sure the community understands what the changes would mean. This idea, however, was not universally celebrated.
Even so, I urge the school board to set a time and place to tell the community definitively what the revised project includes (and doesn’t) and how much it’s expected to cost. I follow these things more closely than most residents, and I have no idea what the project now represents. (If you’re on the school board and believe that you’ve kept the community up to date, please believe me when I say that, from out here, we have no idea what’s going on.)
So there you have it: progress but also confusion. And, as I predicted earlier, it looks very much like, compared to what the community was told to expect for the project, we must now pay more and get less.
Other coverage:
Update: another briefing on the high-school project is scheduled for next week at 6:30 pm, before the regular school-board meeting on Monday, June 20, 2011. The meetings will be held in the high-school library.
Before last night’s school-board meeting, there was another update from the architect and construction manager on the effort to reduce the price of the high-school project. The meeting was scheduled to run for 30 minutes; it ran for about two and a half hours.
There are too many details to recount; if you want them, listen to the meeting recording over at Lebo Citizens. But here are some interesting things I observed.
First, there is evidence of progress: a list of about 200 options for reducing the project’s cost. I think I heard it said that the list represents a potential $14-million savings. That’s the good news.
The bad news is that there is evidence that something’s not adding up. School director Dan Remely noted that the project’s cost per square foot is still unusually high. Even with the proposed savings, we’re only saving a couple of dollars per square foot, he noted.
The project was originally estimated at about $185/sq-ft but the actual bids put it at about $220/sq-ft. This graphic, which I prepared in January, 2010, shows how unusually high $220/sq-ft actually is. In short, 90% of similar projects from 2008 cost less than $150/sq-ft. Our project costs that and nearly half again more.
Further, School director Ed Kubit shared some of his back-of-the-envelope calculations and said that, if the proposed cost reductions were accepted and had their intended effect, their savings would just reduce the project’s cost to the previously declared maximum of $113 million – if applied to the lowest bid we got in April. He pointed out, however, that we might not see bids that competitive in the next round of bidding. The margin for error was slim.
One thing that concerned me was that head architect Tom Celli “urged” the school board to consider switching from single to multiple prime contracts. The general understanding is that this switch would reduce the construction bids – making it more likely that a bid or two would squeak under the maximum – but, ultimately, cost the community more during actual construction. Most school directors, thankfully, seemed to be skeptical if not resistant to this advice.
Another thing that concerned me is that it wasn’t clear what effects all these proposed changes would have. School director Lawrence Lebowitz said that he was concerned about “what is coming out” of the project. Other directors echoed his concern, saying that they didn’t even know what the exterior of the building was going to look like any more. The architects said they could furnish “elevations” to help the school directors visualize the changes, but a 3D computer rendering would take too long to prepare.
If you’re getting the picture that nobody seems to have a handle on these changes, you’re seeing what I’m seeing. Some directors see it, too. I heard at least two say that the timeline for considering changes was too fast. They felt rushed and were concerned that, if they had a hard time keeping up, the public was probably even further behind. Two directors said that there ought to be a public forum to make sure the community understands what the changes would mean. This idea, however, was not universally celebrated.
Even so, I urge the school board to set a time and place to tell the community definitively what the revised project includes (and doesn’t) and how much it’s expected to cost. I follow these things more closely than most residents, and I have no idea what the project now represents. (If you’re on the school board and believe that you’ve kept the community up to date, please believe me when I say that, from out here, we have no idea what’s going on.)
So there you have it: progress but also confusion. And, as I predicted earlier, it looks very much like, compared to what the community was told to expect for the project, we must now pay more and get less.
Other coverage:
- Architect's Update (Lebo Citizens blog)
- Mt. Lebanon trying to get high school renovation costs down (Pittsburgh Tribune-Review)
- Mt. Lebanon board reviews renovation costs (The Almanac)
13 Comments:
Tom,
Excellent post ! This project is fast approaching *theatre of the absurd* status.
I recall back in March, 2010 at the Zoning Hearing Board hearing, Celli & Steinhauer testifying under oath that the project could not be reduced in size or design features because it had to conform to the 15 project design criteria in order to meet the programatic needs necessary to achieve a 21st. century education....the District was committed to provide the very best education possible.
We have also been told along the way with each cost estimate update that the design had been carefully "value engineered" for cost control.
Given all this, how is it that now, suddenly :
1) the size of the HS can be reduced by 33,000 square feet ?
2) there are now 197 individual actual & pending cost reduction items being considered ?
3) that a number of items can be removed from the official project entirely and be treated and funded separately later as Capital Projects....and that several of these are really only *maybe needed* items cause the jury is still out as to whether they are in fact really needed, necessary or even work for that *required 21st. century education* ?
4) and this will not affect the defined programatic needs, or seemingly one concludes a 21st. century education ?
Dan Remely actually did a standout performance in this *theatre* episode, he sincerely did, and should be applauded.
As he has done previously along the way, Celli is pressing the SB to make hurried decisions without benefit of all the necessary facts. This time in order to meet a date set by who knows of September 2nd. for completed documents for rebid...he needs their answers by Thursday.
The curtain rises once again in a week for the next act.
Bill Lewis
Bill, thanks for reminding about next week’s update. I have revised the article to let people know that it’s at 6:30 pm on Monday, June 20, 2011 in the high-school library.
My impression of the meeting was many board members did not understand the project at this point but continued to micromanage their consultants recommendations rather than taking their advice.
John Ewing
The cumulative distribution function of costs per square foot looks like there are only 13 or so data points. So why not just publish the data in a table? That way we might see if we are comparing apples to apples. Which project is the one that was higher and why? And why should we care about the ones that are so much lower? Just lumping the numbers together doesn't say much unless there is a vastly higher number of data points. I think it's much better to compare directly to other South Hills projects like Bethel Park and Baldwin and forget the rest.
David Brown,
Thanks for your questions. Here are the answers.
Why present the data as a graph instead of a table?
Because graphs make data easier to interpret than tables, especially when you want to understand relationships and make comparisons, which I did. (For a good introduction, see Gelman et al., “Let’s practice what we preach: Turning tables into graphs”.) Nevertheless, if you want the tabular version of the data, here’s the original data set, as I received it from the Pennsylvania Department of Education. (Note that I did not include the new-construction rows in my plot of the empirical CDF because I wanted to compare our renovation/reconstruction options with other renovations and reconstructions.)
Which project [of those compared to ours] is the one that was higher and why?
Actually, none of those projects were higher; our bids came in at around $220/sq-ft. If you draw a vertical line at $220/sq-ft on the plot, you’ll see that it’s beyond the most-extreme point on the curve. (The vertical red line on the plot represents the cost per square foot not of our actual project but of the “$103-million Band-Aid” option that the school board rejected for not doing enough.)
And why should we care about the ones that are so much lower?
We should care about the lower ones because their existence is evidence that the higher ones are higher. And, therefore, when every other project, after adjusting for size, turns out to be less expensive than the project we’ve chosen, that relationship is evidence that we’re overspending; the more projects that come in below ours, the stronger that evidence. Now, there may be good explanations for our project’s extreme costliness, but the more extreme our project is, compared to all the others, the better those explanations had better be. That’s because those explanations are competing against an alternative explanation – that we’re overspending – and that explanation gets easier to believe as our project’s price tag becomes more extreme.
Just lumping the numbers together doesn't say much unless there is a vastly higher number of data points.
Do you truly believe this? If there were only 14 students in a class, could we say nothing useful about their grades? If there are only a small number of school renovations each year, does that mean we can make no meaningful inferences about school renovations? After all, there will never be “vast numbers” of them.
When our project is (literally) off the chart that represents a typical year’s school renovations in Pennsylvania, it’s evidence that something is unusual about our project’s cost. Yes, we have only 14 comparable data points, but that’s the actual distribution for 2008. Do you have any reason to believe that school construction in 2008 was unusually inexpensive and, therefore, that comparing our options to the 2008 distribution makes them look unfairly extreme? (The Turner Building Cost Index suggests otherwise.)
I think it's much better to compare directly to other South Hills projects like Bethel Park and Baldwin and forget the rest.
I think it’s much better to use all the evidence we have. If you want to weigh those geographically closer bits of evidence more heavily, fine. But to “forget the rest” is to dismiss evidence that’s likely to be informative. And that’s not a good way to improve the correspondence between one’s beliefs and reality.
Cheers,
Tom
Hey Tom,
Thanks for your response. I have a couple of follow-up points to consider:
1. I said "so much lower", not just "lower". You argued against the latter. My point was that there's not much you can say about the ones that were "so much lower" other than simply that they were lower. It is likely that any direct comparison of scope would reveal important differences that make simple comparison of the bottom lines pointless.
2. It looks like you divided structure cost by square feet. Since we have a mix of new construction and alterations it seems reasonable to include the new construction data points as well. What would your CDF plot look like without excluding most of the projects with higher costs per square foot?
3. I don't have the bid tabs, but how were you able to get just the structure cost of our project? Don't our bids include site development and fixtures?
4. I guess 2010 data would be asking too much, but do you have data from 2009? Where would our neighbors fall on this graph?
5. As for graphical versus tabular, when graphical presentation illuminates the information of course it is preferred, but when it is used to hide important detail I don't think even the authors of the paper you cite would approve.
All the best,
Dave
David Brown,
Thanks for continuing the conversation. Before I answer your questions, though, I’d like to bring us back to the claims my graphic supports. Rather than quibble about the graphic (although I’m happy to; I may learn something), I’d appreciate it if you would just straight out tell me whether you disagree with my claims and, if so, explain what makes you believe they’re false.
Claim 1. Back in 2009 and 2010, when the school district’s architect offered the $103-million option that was generally represented as the least-expensive option available to us, and when the school district considered that option, there was reason to doubt that that option was truly the least-expensive option available to us. (That reason is summarized in my graphic.)
Claim 2. Our current renovation project was just bid at about $220/sq-ft, and $220/sq-ft is unusually expensive.
What do you think? True or false, and why?
Now, to your questions:
1(a). No, I argued for including all relevant evidence. The lowest points are relevant because they tell us just how high the high points are.
1(b). If you believe that a direct comparison would yield evidence that would challenge the belief that our project is unusually expensive (after adjusting for size), compare away. I’d love to see what you come up with.
2(a). I didn’t divide by anything: the Pennsylvania Department of Education did. I just took their cost-per-square-foot figures, as reported, and plotted them.
2(b). Why do you believe it’s reasonable to compare our project to NEW construction projects? (A “mix of new construction and alterations” is exactly what the ADD/ALT category represents, doesn’t it? And that category is already included.)
2(c). Even if you do include the NEW category, $220/sq-ft is still in the top eighth of the distribution. Again: unusually expensive.
3. I didn’t compute our project’s cost per square foot from the structure costs; I used the school district’s cost-per-square-foot numbers directly (e.g., the $220/sq-ft was mentioned at the architect’s update last Monday). I’m assuming that the school district follows the PDE’s standards for computing and reporting these things. (Further, since Dan Remely called out the unusually high cost per square foot at the update meeting, I doubt he would have reached his conclusion about the cost by comparing apples to oranges.)
4. Getting that 2008 data from the PDE was an exercise in frustration that I don’t care to repeat. (The PlanCon process, I learned, as of January, 2010, was still mostly paper based.) If you want to get more data yourself, feel free. I’d be happy to plot it or to enjoy your plots of it. In fact, I’d love to see that data because I think that late 2008 may have been when construction costs peaked and, therefore, that comparing our costs to 2008 costs may understate how unusually expensive our costs really are.
5. If you believe I’ve attempted to “hide” any “important detail” in my plot, please be specific. (Remember, I created that plot, back in January, 2010, to show something about the $103-million pure-renovation option that had been rejected by the school board. Still, if there’s some way I can make my plots clearer or more consistent with reality, I’d appreciate your specific, constructive suggestions.)
Cheers,
Tom
Hi Tom,
Sorry I didn't get right back to you on this. I thought about your questions a lot over the weekend and this is my answer:
1. Project Cost
Recently I spent a long time preparing a very detailed proposal to a client listing everything that is going into their project and all the reasons why various decisions were made, and the client came back to me in such a way that I could tell all they looked at was the price on the last page. That didn't earn my respect, and so I'm not willing to behave that way when I'm on the purchasing side.
Obviously the cost of the high school project sounds high, but I just don't know: (a) what specific things our school needs, (b) what is specified in the project plans to meet those needs, and (c) whether the bids for those plans are higher than they should be. Nothing I have read here shows me that you know those things either, fancy graphs notwithstanding.
To me, a valid analysis of the project would have to go into much greater detail than I have seen on Blog Lebo, or anywhere else for that matter. And I don't think either of us has the time that would be needed to perform a proper analysis. I participated in elections of school board members and I consider that to mean I delegated this work to them. I believe they have spent the time required to know these things, subject to themselves delegating many subtasks that would not be appropriate for them to self-perform.
So, I am content to let them manage this project on my behalf - for better or worse. I might have some complaints here and there, but one thing I am not willing to do is criticize them solely based on the single bottom-line price tag that was received.
2. Statistics
One thing I do know is statistics and I can tell when a very simple verbal point is gussied up with statistics to make it look more substantial than it is. It's clear to me that the point you were trying to make was no better or worse with the graph you produced than without it. And, in fact, when I looked at the numbers, your point was not nearly as clear cut as you presented it. I understand you also have a technical training and so should be able to reach the same conclusions. You'll be more convincing to me when I see you presenting all sides of an issue, including those that do not support your thesis. You can carry more when you use both hands.
Best regards,
Dave
David,
Thanks for your response.
Regarding your beliefs about point 1, “Project Cost,” you wrote, “Obviously the cost of the high school project sounds high, but I just don't know: (a) what specific things our school needs, (b) what is specified in the project plans to meet those needs, and (c) whether the bids for those plans are higher than they should be.”
That’s the point: nobody, not even the people we trust to know (a) and (b) and (c), seem to be able to explain the project’s seemingly extraordinary cost. It’s not just you.
You say that you’re content to let the school board manage the project on your behalf, but even they don’t understand why the project is so expensive. They were flabbergasted by the bids, which came in $15 million higher than their price cap and $25 million higher than they expected only days before the bidding. Their hired experts couldn’t explain the staggering price, either.
Nevertheless, it is possible that the school board understands the project’s cost and understands how that cost is justified by the project’s educational benefits. But, if that were true, wouldn’t you expect the school board and its hired experts to actually know what the project costs? Wouldn’t you expect them, in public meetings, not to say in exasperation that they can’t understand why the costs are so much higher than even they expected?
What, then, do you think explains what we have observed over the last few months?
Now, I know it’s possible that there are explanations that would make all these seemingly concerning observations unconcerning, but is it likely? Further, until we have those explanations, what should we believe? Do we truly need a “valid analysis” to be able to say that the project is probably not going as expected? That the project is probably not going well? That we, as a community, are probably going to pay more and get less than we were led to believe?
Surely we can infer something from the evidence before us, can’t we?
Regarding your comments on point 2, “Statistics,” I was hoping for specifics, something concrete that I could change about my plot or my claims to make them more consistent with reality. Since you say you know statistics, can’t you offer more than your general sense that something has been “gussied up”? Can you tell me, for instance, what specifically about my plot or the accompanying claims is inconsistent with reality? Can you tell me how the caption fails to add meaning to the plot? Or how the plot fails to add meaning to the caption (or to the claim I made about the current project’s cost in the main article above)? Can you tell me how the combination of the plot and the caption (or the claim) is not more informative than either alone? Further, can you tell me how the combination somehow misrepresents reality or is unhelpful at understanding reality?
Likewise, you claim that “[my] point was not nearly as clear cut as [I] presented it,” but failed, again, to offer specifics about what makes you believe so, offering only that you don’t believe I’ve presented “all sides of the issue.”
But what’s the other side of the issue I should be presenting? Isn’t that other side, basically, the evidence that the school board is in control of the project and its costs? I don’t have that evidence. Do you? If nobody has it, what do you expect me to present?
David, just level with me: Do you believe that this is what a project looks like when it’s going well?
Cheers,
Tom
If I may.
(and if you guys would rather I not, let me know)
I think the point Mr. Brown may be trying to come at, Tom, is that while it's clear that our project is at the high end of the list, what isn't clear are:
a) what a "typical" bell curve of per-square-footage costs looks like
b) what the list of factors are that can drive any given project towards one end of the bell curve, and
c) quantitatively, how much cost inflation our project has within the cost factors we are facing.
I think Dave's point may be simply: we don't have clear evidence that the most contributory cost factors we have are reducible or removable.
We keep hearing things like: site plan (terrain & logistics), cost environment (materials & the economy), phasing (materials & logistics, again) as well as value-engineering (materials), scope (materials), and aesthetics (materials).
The first list, by and large, seems more tightly defined or even uncontrollable. The latter are a bit more flexible within the project definition.
What I have yet to see is an analysis that puts a fixed cost or a cost multiplier against each of these, as a comparative to "the average project."
If someone says "oh we have a really unusual site plan" - ok, how unusual? 2x? 5x? We could try to find examples of pairs of projects out there that differ largely only in site plan and find out what that multiplier might be.
If we could rank these factors, both in cost and in flexibility, we could well see that the only wiggle room comes in with the aesthetics and scope elements. But if the multipliers for those items just really aren't much of a relative cost contributor, it's not going to count for much.
And by all means: board, MDT, anybody, if you have this data and are using it to make these decisions, please publish it!. We are past the handwavy-assertions-of-diligence stage!!
Matt,
By all means, jump into the conversation. You have a great way of getting to the heart of matters.
I think you’re right, both about Mr. Brown’s probable point and about our incomplete knowledge of the high-school project’s particulars and the distribution of projects in general. I also agree that it’s possible that among these unknowns are factors that could make the project’s seemingly extraordinary cost seem reasonable.
My point is that we don’t need complete knowledge about any of these things to be able to make informed inferences that are likely to prove correct.
Looking at the 2008 data, for example, ought to give us some understanding of the “typical” distribution (I hesitate to call it a bell curve, although it does appear to be mound shaped). Doesn’t this distribution tell us something about the influence of factors known and unknown? Knowing that our lowest-cost option and our current option are both in the right tail of this distribution and other cost distributions, such as those behind PJ Dick’s estimation models and Mr. Remely’s concern that something was wrong with the costs, ought to tell us something, too.
To be sure, it’s not enough for us to be able to draw conclusions with certainty, but this knowledge, however incomplete, ought to push our beliefs one way or another.
I agree that if we could rank the influence of the relevant factors we could make more-informed judgments, but we don’t have the knowledge to perform that ranking. Given what we do have, then, what can we say?
Cheers,
Tom
We absolutely can conclude several things from this data.
I put together a histogram of the projects in $25/sqft bins. You're right, it's not a bell curve, but it does tell a story.
Check out the graphs on tab 2 of my spreadsheet
At a suppositional figure of $220/sqft, that puts our project at 57%, or 1.18 standard deviations, above the mean of all projects, and 108% (double!), or 2.7 standard deviations, above the add/alt projects. Check out tab 1.
So, yes - this particular turkey is a relatively expensive turkey. Especially among mixed construction/renovation projects, which skew significantly cheaper (~$109/sqft).
So - back to my prior comment: to me the next question is - why? What's up with the Upper Dublin SD (new HS) [uh, wow], Wilson SD (new MS), and
Owen J Roberts SD (add/alt MS) projects?
Why are they outliers? What drove the costs? Are they built off the sides of cliffs? (google say: no!) Were there major logistical challenges? Or are they just gold plated?
Can we maybe get a hold of their plan con materials or even reach out and touch someone to get the nutshell?
(And hey, did those Springfield SD people get blinkered or what?!)
Matt,
I like where you're going. (Although, for data sets like this, I like using empirical CDF plots because they avoid the binning problems inherent in histograms and make shares easy to compute.)
My short take on the data so far has been that this turkey is extraordinarily expensive, and therefore requires extraordinary evidence to support the belief that it's really a normal bird, once you get under all the feathers.
Like you, and like some of the school directors, I'm still waiting for that evidence. I'm not talking about vague statements that there are "site challenges" and such. I'm talking about something that quantitatively connects the dots between the claims of the project's supporters and the reality of the recent bids.
Nobody seems to be able to reconcile our current understanding of the project to the $130-million-plus project cost implied by the bids.
That's concerning, isn't it?
Cheers,
Tom
P.S. For those who can't see Matt's histograms, here are purely textual stem-and-leaf plots showing the 2008 project-cost distributions.
First, the distribution of costs per square foot for renovations and mixed renovation/new projects. (This is the distribution which we would expect to be more applicable to our project). Here it is:
> stem(pde2008_nonnew$Cost.per.Square.Foot)
0 | 6678
1 | 00011123
1 | 5
2 | 1
And the distribution of costs per square foot for all projects, including new construction:
> stem(pde2008$Cost.per.Square.Foot)
0 | 6678
1 | 0001112334
1 | 58999
2 | 01124
2 | 9
In both plots, the decimal point is 2 digits to the right of the |. The 2 | 1 in the final line of the first plot, for example, represents $210/sq-ft, the cost of the most-expensive project in the distribution of renovations and mixed renovation/new projects.
Cheers,
Tom
Post a Comment
<< Home