[ncdnhc-discuss] [long] The NCDNHC's .org report is numerically inconsistent.

[Milton Mueller]

>>> Thomas Roessler <roessler at does-not-exist.org> 08/20/02 04:48PM >>>
>Well, the table on page 49 is in fact a bit worse than just that:  
>The names column is sorted by the actual final score.  The  
>responsiveness column is sorted in decreasing order (ups, not  
>precisely - neustar and register.org fall out of that pattern - ok,  
>it's just not sorted in any sensible way at all ;). 
  
[this stuff goes on for two more paragraphs]

The p. 49 table is not sorted at all, as I said in my last letter.
I'm not sure why you enjoy expressing your confusion about
an unsorted table that was not meant for public display 
at such length, but I doubt if the rest of us are interested. 

You have one thing of relevance to state here.
So far, you have identified two typographical 
errors in the presentation. Neither of them affected the 
results, because the total values were not in error,
only the component values. Thanks for that.  
Now let's move on, ok?

>>ISOC really is a 21.25, check your arithmetic. I think you made 
>>the arithmetic mistake this time ;-)

>I don't think so (I hope this makes it to you in reasonably readable 
>form):

>ISOC	3	3	5	5	3	5	2

There's the mistake: the first number is a 2, in the ACTUAL table;
i.e. the Excel spreadsheet. You could have figured this out from my
last message. 

>Now, for the "public support" category (see my blog for a much more  
> - possibly too - polemic version of this): 

Yes, you seem to have completely lost sight of the forest 
for the trees. We spelled out the limitations of our
method very clearly, and these can be taken account of 
in making a judgment. 

When the dust settles, your points about public support 
do not seem to be very intelligent to me. Certainly they
do not justify the noise you are making. Let me go 
through them.

>The only source you can draw from for public support 
> are the postings to ICANN's .org forum. 

Yes, exactly. We counted endorsements. 
As we explicitly noted in the report, that was
the only form of evidence available. And ICANN's
RFP made it clear to bidders that they would be
expected to supply such expressions. (I think
we have seen enough of unilateral divinations of 
"consensus" in the ICANN space to know that we 
want to rely on tangible evidence, not someone's
idea of what is "representative.")

> The problem with this is that it's  
>a classic, self-selected (or, even worse, orchestrated) survey.   
>What do these numbers really tell us?  The answer is: They only tell 
>us how good the various proponents were at mobilizing their  
>respective "fan clubs".  

Yes, indeed. But ability to mobilize is real.
if you cannot get any people, or organizations,
to spend a half an hour writing a letter and sending
it in to ICANN, how much support can you really
claim? Try it some time, if you don't agree.

The alternative you implicitly offer is to divine how
"representative" of .org registrants the support expressed 
for each bidder is. Please tell me how to do THAT objectively?
The only objective method would be to spend 
several million dollars and a year to conduct global surveys of 
noncommercial .org users based on statistically representative 
samples. Most of whom would be unaware of the whole process 
of .org reassignment. 

>Thus, this is, from the very beginning, the worst and most  
>insignificant input you have.  

Calm down. We were ASKED for this estimate
and indeed it was a part of the RFP that all bidders
were asked to provide. If you don't like the fact that the RFP
included this criterion you should have complained back in
April or May, when it was formulated. And I don't agree
that it is insignificant, either. The top tier bidders demonstrated
widespread public support, the others did not. There were
real and significant differences.

>To make it still worse, you have to  
>_estimate_ the number of ISOC class B responses on page 23, because  
>you can't reasonably make the distinction between class As and class 
>Bs for this application (which may indicate that the distinction was 
>the wrong approach to solve this particular problem).

We can and do make a clear distinction between Class A's and B's.
What we can't tell from the evidence is whether the people who
added their name to the Class B list were ISOC members or not. 
Of course, we could have thrown out all the endorsements for
ISOC. Or we could have counted all 500 of them. 
Do you think either would have been fair or accurate? 

===

All of your comments below pertain to parts of the
report that I either did not write or do not agree with
and worked against. So I will let others respond to them
if they feel like it. Bear in mind that this report emerged 
from a committee, composed of different people with 
different ideas. 

>But for the "score-based" approach, you then use some kind of  
>(supposed to be) pseudo-logarithmic rating with additional weights,  
>described on page 43.  There is no rationale for this, and it leads  
>to interesting results.  Just look at the assignment of applications 
>to tiers: In the score-based approach, GNR (like Neustar) has a  
>score of 3, but it's in the C tier (Neustar is in B), while dotorg  
>foundation has 1, and is in the B tier.  That doesn't make sense.

>Finally, one question which doesn't have anything to do with  
>numbers: I have looked a bit at your evaluation of the probable  
>"winner" of the entire process, ISOC.  In the "responsiveness"  
>category, you write:

>ISOC proposes a number of very innovative services designed to  
>respond to the needs of noncommercial entities, not just  
>registrants generally. ISOC therefore received a High rating in  
>this category.  Finally, the Committee notes that although it has  
>made no commitment to support good works, profits from the  
>registry will go to ISOC.  On the arguable proposition that  
>support for IAB/IETF standards processes constitutes good works we 
>awarded ISOC a Low ranking in this category rather than a None. 

>I'm sorry, but I fail to find these services.  I find services which 
>are generally useful for registrants, and services useful for IP  
>owners.  But none specifically targeted at noncommercial entities.  
>Maybe you can shed some light on this?