so the craziness of this site (w/o tech to manage it to infinity.. fitting to use the word manage here.. perhaps only managers we need are ones not biased..).. adding david per nicholas rec.. but haven’t added nicholas.. who has been a huge influence.. just pointing that out.. nothing complete/finished/obvious/measurable.. like on people page.. often those who influence most don’t show up on site..
adding page this day:
(first part .. on measuring.. posted first on measuring things page.. then when nicholas sent me more from david.. on managing/corps.. decided to add his own page)
via @ultimape stream –
n: I never want to view you as some generic consumer. Imagining at eating my shit(posts) makes me laugh… & reminds me you’re human. Be human.
m: perhaps being human.. begs we quite measuring exchange..
n:There could be something to it. I’d love to know how many people i’m causing pain to, so I can avoid it if I can.
m: not talking about you.. causing pain.. talking about the system
n: That’s what inspired this. I’m thinking about advertising incentives and how it influences ‘the system’. :)
n: But those kinds of measurements have to be rooted in empathy first, or you miss it. ribbonfarm.com/2016/06/09/goo…
he shared (by David): https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-measurement-is-hard/
m: drucker’s – If you can’t measure it, you can’t manage it. – perhaps we’re not supposed to manage.. it
m: ‘good metrics let us make rules that create the right incentives to improve’ – not sure incentives are human natural
n:Agree. I like this framing – reminds me how absurd some of this sounds (and how it probably is): twitter.com/visakanv/statu…
n:Seeing the marketing language juxtaposed with the human conneciton makes me think there’s two competing worlds of thought.
m:or perhaps one we could/should disengage from
n:twitter.com/ultimape/statu… Become an expert in the system you want to destroy. In a perfect world, there would be no such thing as marketers. Just cool people sharing.
n:My focus question: how to destabilize in such a way that we don’t impoverish people in the process and lead to great suffering?
m:am thinking it’s gotta be a leap (to a nother way) .. since we now have the means for that
back to goodhart’s law post:
As with many other cognitive skills, the fact that it’s a counter-intuitive learned skill for children means that adults don’t do it intuitively either. So why are measures used? There are lots of good reasons, and I think a useful heuristic for understanding where to use them is to look for the triad of intuition, trust, and complexity
Measurement replaces intuition, which is often fallible. It replaces trust, which is often misplaced. It finesses complexity, which is frequently irreducible. So faulty intuition, untrusted partners, and complex systems can be understood via intuitive, trustworthy, simple metrics. If this seems reductive, it’s worth noting how successful the strategy has been, historically. Wherever and whenever metrics proliferated, overall, the world seems to have improved.
Managers Need to Measure
do we need managers..?
i’m thinking only at the ginorm small level.. and now we’re back to more of a gut.. in the moment measure..
after i tweet/ask that.. Nicholas tweets this
[that’s when i decided to add page.. i talk to the go corp or go home post below]
The key problem of using a measure as a metric is sometimes referred to as Goodhart’s law. The law is based on a 1975 paper on economic regulation, and is typically paraphrased as “When a measure becomes a metric, it ceases to be a good measure.” When managing a system, this problem is critical, but is deeper than it at first seems, as I’ll argue.
I think these can be understood via the three key things measurement can replace, mentioned above: intuition, trust, and lack of understanding of a complex system.
Hubbard notes, it is precisely the areas where no measurement has been done before that his method typically finds tremendous value in creating metrics. He opens the book saying that “no matter how ‘fuzzy’ the measurement is, it’s still a measurement if it tells you more than you knew before.”
also makes me think of dreyfus – an ai (or whatever) based on a false premise..
A kid tells you they are the smartest person in their freshman college class. You don’t know them — but their SAT score can tell you if their claim is plausible
what the heck..
Statistics, like any other discipline, is a way to think, but it can’t stop you from lying to yourself, or to others. It can, however, prevent others from lying to you — but only if you *trust the source of the data, and understand and trust the methods they used.
deeper – what the data is.. ie: sat data isn’t a measure of how smart a person is.. (of course.. if your defn of smart is successful at sat tests.. then ok)
But if we want good metrics, we probably need to collect data — and that is hard to verify from the outside. If someone who produced or has access to the numbers is motivated to fudge the numbers, the dishonesty is difficult to detect.
begs a do over.. and we focus on self talk as data
Perhaps the best way to mitigate the risk of dishonesty is to adopt the strategy testing companies use: create trust through disaggregating responsibility. Test takers are monitored for cheating, graders are anonymized, and the people creating the test have no stake in the game.
best safety/guard: gershenfeld sel
Complex systems have complex problems that need to be solved. Measures can summarize, but they don’t reduce the complexity. This means that
measures hide problems, or create them, instead of solving them .
1\First, metrics reduce dimensionality —
the single number of a metric doesn’t represent everything in the system, leading to a loss in fidelity.
2\Second in our list of difficulties is causality. Defining causality is contentious, but I’ll try to keep it simple. A causes B if when we magically manipulate A, but nothing else (even things that would normally change), B changes. This is made harder in practice because usually
third problem: metrics are usually reified….In some sense,
your brain finds it easier to create a non-existent object than to fail to recognize a pattern
What’s harmful is that when we create a measure, it is never the thing we care about, and we always want to make decisions. And if you reify metrics away from the true goal, you end up in trouble when they stop being good measures.
Systems using measures are incentivized to perform certain ways – they self optimize.
Metrics make things better overall, but only occurs to the extent that they are effective at encouraging the true goals of the system. To the extent that they are misaligned, the system’s behavior will diverge from the goals being mismeasured. And once the system diverges, the very incentives you put in place make it hard to change. The problem with Goodhart’s law is that it is impossible to get metrics exactly right, and so the pressure of the system will always warp until the metrics diverge from the actual goal.
Specify a metric for user engagement, and as Zeynep Tufekci pointed out in a very worthwhile analysis, Facebook starts to select for sensationalism and garbage. In the article, she says this is because algorithms are not neutral — but I think she’s wrong. Tools themselves are neutral, but how they are used are not.
agree.. that’s why ai is messed up.. we keep using tools wrong way.. we’re missing such an incredible opp.. ie: we let tech/mech listen to all the voices.. since we can’t.. making ai – augmented interdependence..
Complex systems can only be managed using metrics, and once the metrics are put in place, everyone is being incentivized to follow the system’s logic, to the exclusion of the original goals.
? perhaps complex systems shouldn’t be managed.. esp if begs incentives.. to me that’s a sign .. not deep enough..
from Nicholas share above – on David exploring why corporations happen:
Even startups can’t stay startups. Github, the catalyst for distributed software companies everywhere, is itself restructuring. As the author of this post on Github’s restructuring puts it, “Out with flat org structure based purely on meritocracy, in with supervisors and middle managers.” But why?
My basic argument is this: the legibility that lets companies scale is at odds with the flexible way typical startups operate.
perhaps assuming legibility (the quality of being clear enough to read) is the problem/issue..
Meritocracy, a rallying cry for the Silicon Valley startup mindset, only works when merit can be seen and rewarded by management.
perhaps assuming meritocracy ness is killing us
Merit can only be obvious to everyone when groups are small enough. Once Github passed Dunbar’s Number, there was going to be no way for people to work as one coherent culture — though they grew so fast they reached double that number before the VCs put in someone to bureaucratize and let them scale.
thinking jo freeman ness.. and us missing the point – one point being.. it’s not about merit.. it’s about connecting (per curiosity et al).. and so it’s not impossible beyond a certain size 7, 15, 150, whatever…. we just have to org it different..
perhaps the impossibility ness goes away when we disengage from the impossible claim to be legit merit ing..
Walmart need to be really efficient at doing its single job, selling products that it buys cheaply, at scale. Adapting might be hard, but the efficiency of having clear reports and being able to optimize revenue per employee is worth it.
if our focus is business.. but not if our focus is people.. medium/message matter little if wrong focus
The big companies have legible, normalized datasets. And the legible data needs analysis once it gets larger than a person can fit inside their tiny 7+/- 2 item brains.
And the lack of legibility
hasn’t hurt me none,
’cause I scribble diagrams on the wall.
Rigid structure doesn’t allow for rapid pivoting, but scribbling on the wall does. The pivoting that startups go through lets them find a niche and build a culture that demands investment into new, risky, and possibly *profitable ideas.
profitable as goal.. as anything.. is compromising us..
They thrive on the flexibility that illegible structures permit.
If data describing the organization is *small enough, it doesn’t need to be legible to be understood
*ginorm small is what we need.. to be able to dance.. and esp important.. to be able to dance our authentic/idio dance (which merit/measuring messes with.. compromises our curiosities and steals our time) ie: hlb
the flexibility of illegibility is fantastic.
let’s go with that.. we now can.. with tech.. let’s go for the antifragility ness of us
*Flexible meritocracy is easy when you can shift people around as soon as you see they are ready without needing to rely on metrics identifying top performers, or wait for yearly performance reviews and promotion cycles. But this changes as a company grows — probably around the time that management notices it doesn’t know who everyone is.
*may be easy when small – but is it ever the right thing..? for humanity..?
How do you store the org chart data if the relationships are unclear?
maybe by hosting-life-bit
Iain M. Banks’ “Culture” series, in part, explores a society without the first two levels. There is no central anything, and everyone does whatever they want. But in such a culture work is voluntary, and rare.
they aren’t doing whatever they want.. we can’t until we all can.. partial ness doesn’t work because of our interconnectedness.. we keep basing our potential on experimentation/findings that start out with assuming ie: science of people.. aka: not us.. as subjects in the experiment..
Flat unstructured meritocracy works!
Unfortunately, as I noted earlier, this doesn’t scale
not because of the flat ness or invisible/illegible structure.. the meritocracy ness is the killer
I agree that merit is the critical question, but both observationally and theoretically it falls apart with larger size and lower legibility
and i back to him
perhaps current observation/theory irrelevant if we disengage from it (merit ness)
If we tried, it would look, at best, like a high school’s social scene. You’d see cliques, relationships that form and dissolve rapidly, and little if any productive work being done, at least by the majority of the students.
It’s still not enough to help our epidemiologists, or enough to explain the problems with startups scaling. High schoolers, like those in our study, are more complicated. Obviously we need a more flexible, less legible structure to let us represent and understand their relationships.
What I’ll call pivot culture, which exists in high schools and colleges, doesn’t want or need legibility. But if you’re a lawyer, you need to know who inherits, who pays child support, and who gets hospital visitation rights. Tradeoffs exist between legibility and the freedom of arbitrary structure — so it’s a good thing for lawyers that as people grow up, they decide on more legible relationships.
Flat meritocracies are awesome. Can’t an emergent startup culture, full of collaboration and creativity, allow companies to succeed without turning into corporate bureaucracies? To phrase this differently, Peter Pan has more fun, and startups don’t want to grow up. Can’t kids stay kids, and be successful too?
No. This is where the social graph becomes critical. The number of possible social graphs explodes very quickly; 7 people have only 156 possible configurations, 10 have over a quarter million, and by the time you get to 15 people, the quadrillions of possible structures is *clearly unmanageable.
*not by unbiased tech/mech.. perhaps a blockchain type that’s not compromised by measuring/judging ness.. just facil/connect by/to curiosity..
or could say.. yeah.. unmanageable.. as it should be..
This means that decision makers can’t understand the impacts of their decisions. Hiring people becomes a mess, since the only way to scale anything is to disrupt this chaotic network. Firing people, or even reassigning them, is worse — it may be removing a key piece of some process a manager, or even the employee, doesn’t notice.
What is the alternative? Simple, legible org charts.
Why? *Legibility is related to ease of communication: if something is legible, you can see where to go and what to do.
aside from corp of go home post
reply from david:
I’m unsure if there are ways to communicate without legibility, but I’m sure most illegible systems *make communication much harder
*make it harder..? or simply make us have to realize.. communication is never finished..?
what is legible.. who decides..
then i asked him how he’d define legibility and he ref’d me to his other post.. so adding/digging thru that:
from legibility link
The picture is not an exception, and the word “legibility” is not a metaphor; the actual visual/textual sense of the word (as in “readability”) is what is meant.
the reason i asked his defn is because i’m thinking.. what’s getting us is.. readability by who..? thinking we need to get rid of the pre req of prep/training in order to communicate..
” The deep failure in thinking lies is the mistaken assumption that thriving, successful and functional realities must necessarily be legible. Or at least more legible to the all-seeing statist eye in the sky (many of the pictures in the book are literally aerial views) than to the local, embedded, eye on the ground.
I suspect that what tempts us into this failure is that legibility quells the anxieties evoked by apparent chaos.
this is huge.. are we insisting on legibility because we fear uncertainty.. when.. it’s’ the uncertainty that makes us free..
If my conjecture is correct, then the High Modernist failure-through-legibility-seeking formula is a large scale effect of the rationalization of the fear of (apparent) chaos.
The Napoleanic era saw the spread of the metric system; again an idea that is highly rational from a centralized bird’s eye view, but often stupid with respect to the subtle local adaptions of the systems it displaced. Again this displaced a good deal of local power and value, and created many injustices and local irrationalities, but the shift brought with it the benefits of improved communication and wide-area commerce.
The reason the formula is generally dangerous, and a formula for failure, is that it does not operate by a thoughtful consideration of local/global tradeoffs, but through the imposition of a singular view as “best for all” in a pseudo-scientific sense.
there you go.. not even global vs local.. but the idea of algo ness for humanity .. for communication.. et al
the process is driven by a naive “best for everybody” paternalism, that genuinely intends to improve the lives of the people it affects. The high-modernist reformer is driven by a naive-scientific Utopian vision that does not tolerate dissent, because it believes it is dealing in scientific truths.
back to corp or go home post:
the less legible the organization is, the harder it is to be resilient.
is it..? thinking how we gain from dis order
[not to mention.. again.. that ie’s are based on assumptions of ie: business; office; salesman; firing; hiring;…]
We’d love to have flexibility, but the cost is scale, integration, and *profitability.
perhaps the compromise is thinking *profits.. rather then people
The math of complexity isn’t changing, and *humans have cognitive limits. That means we need to accept that growth of companies post-startup phase will not be exponential, nor even linear, but logarithmic — scaling along with the legibility of a tree.
to help cities/states/countries do a better job at protecting their citizens..
perhaps gershenfeld sel.. as best safety ness
rand corp is non profit.. non partisan think tank
thinking Hubert (ref above) working there
what i do there is help inform decision makers.. about what the diff policy options are for measuring catastrophic risks..
help inform decision makers.. good.. diff policy options.. compromising..
trying to encourage cities to do things that will prevent future disasters.. make them more resilient.. to the impacts of future disasters..
i like the idea of being able to do the same type of quantitative modeling.. but for the public good/interest
understood in the context of what it is we were put here to do..
lander college and rand corp – 2015:
We’re trying to encourage cities to become more resilient to the impacts of future natural disasters.