When I was almost finished with my studies, I thought for a long time about whether I should do a PhD. On the one hand, I was in actual love with my master’s research topic and I had the opportunity to continue right where I left off. On the other hand, PhD students lead a stressful life of imposter syndrome and are paid a pittance.

I spent quite some time searching for articles and videos with titles like “why I quit my phd” to see if any of their feelings and thoughts sounded like the sort of things I might think. That was a useful activity which I would recommend to any prospective PhD student.

Right now, I am 2 years into my PhD. The title question remains. Here is a list of things that would have allowed me to make a more informed decision.

- Imposter syndrome starts the day you’re hired and won’t stop for many years to come.
- It is normal to fantasize about quitting every so often.
- You will at some point realize how much more money you would make in industry. That is how much you’re effectively paying to do a PhD.
- You will like your job less than you expected beforehand.
- It is probably better to drown your sorrows in online procrastination than in substance abuse.
- Free weekends will be rare, or at least you will feel guilty about them. Other people will always seem to be working more than you.
- Depending on how good you are at making friends, your first conferences and workshops will be sad and lonely and exhausting.
- Your sense of self-worth will become tied to your sense of how your research is going.
- When your proofs or experiments are not working, you will feel frustrated and miserable for days/weeks/months on end.
- When your theories and data do start working, the world will shine and your heart feels made of clouds and it will be blissful. For an hour or so, until you find the irrecoverable bug in your proof.
- Academia is a scam. The fraction of your time reserved for research is a strictly decreasing function of time. Research directions are chosen because they are uncrowded and occasionally interesting, not because the outcomes would have any relevance to the world at large. Every single postdoc is sad and lonely because they have to move long distances every year or so, preventing them from having any friends.

That said, getting to do a PhD is undoubtedly a privilege. While the whole world is burning, trillions of non-human animals are brutally slaughtered every year for needless human consumption, and 815 million humans are too poor to afford enough food, you get play a part in the Unidirectional March Of Human Progress by spending four years in your safe, rich bubble thinking about a useless problem that Erdős once mentioned. Be sure to enjoy it.

]]>Continue reading "7. Links #2: AI Safety-critical reading list"

]]>So, I started the anti-AI Safety blogging series because I would be a good fit for the cause area as described by e.g., 80,000 Hours and it seemed reasonable to think through the arguments myself. As it turns out, they don’t stand up to scrunity. I decided to keep on writing for a bit anyway, as all AI Risk enthusiasts seem to be unaware of the counterarguments. I thought there was nothing out there in writing. Boy was I wrong.

This is a non-exhaustive list of links relating to AI Safety skepticism. For more, check out the similar reading lists by Marcus Vindig and by Alexander Kruel. Overlap between these lists is minimal and restricted to a couple of **particularly good resources**.

**Rodney Brooks** writes from MIT Technology Review of the seven deadly sins of predicting the future of AI. If you find a paywall, either clear your cookies or view a less edited version on Brooks’ website. His other essays on Super Intelligence are also well-worth checking out.

Wolfgang Schwarz published his referee report of Yudkowsky (MIRI) and Soares’ (MIRI) Functional Decision Theory. I’ll quote a single paragraph, which I think accurately illustrates the whole review: *“The standards for deserving publication in academic philosophy are relatively simple and self-explanatory. A paper should make a significant point, it should be clearly written, it should correctly position itself in the existing literature, and it should support its main claims by coherent arguments. The paper I read sadly fell short on all these points, except the first. (It does make a significant point.)”*

Ben Garfinkel gave a talk at EA Global 2018 titled “How sure are we about this AI stuff?”, calling for EA’s to be more critical about AI Safety as a cause area. Garfinkel knows his audience well, as everything is phrased so as to make EA’s think without ruffling feathers

Oren Etzioni writes in MIT Technology Review about the survey data Bostrom talks about in Superintelligence and offers alternative data that suggest a very different picture

**Maciej Cegłowski**‘s talks are always excellent and “Superintelligence: The Idea That Eats Smart People” is no exception. (via)

EA Forum user Fods12 wrote a five-part critique of Superintelligence. They hit on a number of good objections. The posts sadly got little quality engagement, indicative of both the writing quality and of the rest of the EA Forum’s userbase.

Even transhumanists can be reasonable, like Monica Anderson who writes Problem Solved: Unfriendly AI.

Ernest Davis wrote a review of SuperIntelligence, touching on some of the key weaknesses in Bostrom’s arguments but insufficiently elaborating on each of his arguments. MIRI published a response to the review which I think mostly nitpicks Davis’ phrasing instead of actually engaging with his objections, which to be fair might be the best you can do if you don’t have any better source of exposition on these arguments than Davis’ review. In short, Davis’ review isn’t super good, but MIRI’s response is much worse.

Neil Lawrence critiques Bostrom’s Superintelligence. If I had to excerpt a single representative line, it would be *“I welcome the entry of philosophers to this debate, but I don’t think Superintelligence is contributing as positively as it could have done to the challenges we face. In its current form many of its arguments are distractingly irrelevant.”*

**Magnus Vindig** writes Why Altruists Should Perhaps Not Not Prioritize Artificial Intelligence: A Lengthy Critique, in which he tackles most of the standard EA arguments and points out their hidden assumptions. Topics include, but are not limited to, the incessantly cited AI researcher survey predictions, bad Moore’s law-type arguments, slight-of-hand changing definitions of intelligence, the difficulty of alignment rising for future systems compared to current ones and the enormous experience we have with present-day systems, Instrumental Convergence being under argued, the practical value of being super intelligent. He does not rigorously take down every argument to the full extent possible, but that is probably good because the blog post is 22k words as is. Vindig also wrote Is AI Alignment Possible? in which he argues that the answer is no, both in principle and in practice.

Richard Loosemoore has the right amount of derision that AI Risk deserves, which is different from the right amount of derision for convincing the worriers that they’re wrong. One person who was not convinced is Rob Bensiger of MIRI.

Bill Hibbard has an email exchange with Yudkowsky in which he argues that a Superintelligence would not conflate smiling human faces with nano-scale depictions of such. The whole exchange is kind of predictable and not too informative.

On a related note, Nicholas Agar wrote a paper titled “Don’t Worry about Superintelligence” in which he argues that the first AIs with sophisticated agency are inherently likely to be friendly.

]]>Pokémon Snap first came out today 20 years ago. It is an on-rails shooter game. You’re in a car that drives around an island filled and your goal is to take pretty pictures of the pokémon that live there. It was one of my favourite games when I was little. Its relaxing gameplay and cute jokes still managed to put a smile on my face when I played the game again recently for the first time in years.

It takes only a couple hours to take a picture of every pokemon in the game, though with high variance since some pokemon are really tricky to find. The game feels quite modern despite its age.

At the end of every course you can show a number of photos to Professor Oak and he will rate their quality on a couple different aspects, like how well the pokémon fits in the frame and if they are in a nice pose. The amazing thing is that algorithm almost exactly matches my own opinion of which photos are good. You don’t see that often in algorithms.

The original Pokémon Snap website is surprisingly still online. It is blazingly fast. Wirth’s law is my least favourite law of nature.

]]>You have probably heard the latest AI news cycle, this time the news is about a language model. Let’s forget for a second about the minutiae of how dishonestly the achievement was presented and whether or not the software should be released to the public, other people have done that better than I ever could. I want to point at a cute problem related to authorship and suggest a possible solution.

In a future where most text will be governmental and commercial propaganda written by software, would people want to make sure they only read text written by other sentient beings? Or would at least old people like future me want that? And will that be a rational preference, or just anti-robot prejudice?

Let’s suppose that knowing whether a text is human-written is desireable. How could the future people go about doing that, and can we do anything now to help them?

The interesting case is in verifying that an anonymous self-hosted blog is human-written, or anonymous comments on such a blog. Most other cases seem to be trivially solvable either by having national governments verify digital identities or using interactive protocols (like Turing tests and captchas).

I’ll describe one possible avenue, useable only by anonymous internet users who act now, and possibly only for low stakes. The trick is to generate a pgp key and use it to sign every bit of content you create. The uniqueness of the signature (no two people can claim the content for their own) is established jointly by all internet archiving institutions, who also implicitly timestamp all data. All well-written text from before 2020 has been written by humans, so if you can claim authorship on a good share of that text then you are solid.

]]>Continue reading "Beyond Condorcet winners: Majority Judgement"

]]>There have been rumours on Wikipedia that Michael Balinksi passed away this Februari 4th. If this is indeed the case, then my heartfelt sympathies go out to those around him. I never knew him personally, but he was an amazing scientist and I’ve used results of his more than once. He was quite a phenomenal speaker.

Today I want to talk about a single-winner voting system that Balinksi introduced with Rida Laraki. It is called majority judgement and it is so brilliant that it almost makes me wonder what voting theorists could have been doing both before and since then.

One big concept in social choice theory is majority rule: if most people think that A is better than B, then A should be the winner. Most multi-candidate voting systems generalize this in various ways, always preserving that if candidate A would beat every other candidate B, C, etc, in a pairwise competition, then A should win the election. If candidate A satisfies this criterium, we call A a Condorcet winner. The leading wisdom in social choice theory was that any good voting rule should let the Condorcet winner win (if it exists).

According to my informal sample from the Effective Altruism community, EA’s favourite voting system seems to be approval voting, which is one of these voting systems that generalizes majority rule to multiple candidates.

The genius of majority judgement is that it moves past the Condorcet winner paradigm and considers a perspective beyond that.

To illustrate, let’s assume we’re running an election among two candidates, the red candidate and the blue candidate, and every voter gets a ballot with for each candidate the options “Amazing”, “Good”, “Mediocre” and “Terrible” to express how good of a president they think the candidate would be. Let us for simplicity assume that our voting population consists of 5 groups, the A’s, B’s, C’s, D’s and E’s, and everyone in a group votes the exact same way. The outcome of the election is in the table below.

A | B | C | |

% of population | 40 | 20 | 40 |

Red candidate | Amazing | Mediocre | Terrible |

Blue candidate | Good | Terrible | Amazing |

The red candidate wins the Condorcet-style election: 60% of the population prefers the red candidate over the blue candidate. But the blue candidate is clearly better: 80% of the population considers the blue candidate to be “Good” or better, while 60% of the population considers the red candidate to be “Mediocre” or worse.

Majority judgement is a voting rule designed to have the blue candidate win in the above election. The actual algorithm is a bit involved, but the first step is to compare the median vote: if at least 50% of the voters think the blue candidate is “Good” or better and at least 50% of the voters think that the red candidate is “Mediocre” or worse, than the blue candidate will win. If the median opinion is a tie, a more complicated tie-breaking rule is entered. ((The exact rule satisfies a number of optimality criteria and is the only rule to do so. For this post I want to skip the details.))

I think the concept is very elegant, and I believe that the outcomes really would be better than with a system that elects Condorcet winners. In a talk that Balinksi gave, which I was lucky enough to attend, he pointed out another advantage of the majority judgement rule: it allows voters to express what they think of the candidates. You wouldn’t be asking anyone to “vote for the lesser evil”, everyone can keep their conscience clear. Majority judgement admits a clear way of expressing frustration with both candidates: rank both of them very badly. It also helps that the different options are given in words instead of ranking by numbers, for the latter turns out to entice voters to rate their favourite candidate 10/10 and all others 0/10.

]]>Continue reading "Funding opportunity in academic publishing"

]]>I’ve recently listened to a talk by Jean-Sébastien Caux, the founder, implementer and chairman of SciPost. It looks like a charity that might be worth funding.

Academic publishing is a complete disaster. The incentive structure is messed up. Papers must have more than a certain minimum of content*coolness, but simultaneously they shouldn’t be too long. This results in researchers sticking two unrelated papers together and calling it one thing, and cutting single papers up into multiple pieces to get published. If your result is too simple then it will get rejected so people make their papers more difficult on purpose. There is no pipeline for communicating minor improvements and comments on other people’s papers to the outside world.

Peer review might not literally be a farce but it is closer to being so than anyone is really comfortable with. Because it all happens behind closed doors, peer reviews seldom have constructive feedback in them and reviewers will most likely harm their own careers if they spend time and effort into reviewing that could be spent doing research. People submit everything to the best journals first, trying one rung lower when their paper gets rejected. The review process can take years. Reviews are all hidden from the wider public, so the only way to judge a paper’s quality if you’re not in the field is by looking at citation counts and the journal a paper appeared in.

Publishers make a lot of profit selling the research community’s own results back to them. Journals and impact factors are silly inventions that date back to the dark ages before internet existed and serve little to no use in modern times.

Enter SciPost. Imagine a love child of the free software movement, open access academic publishing, and modern discussion platform design. Submission is free. A manuscript is public from the moment one of the editors decides it is probably worth getting reviewed, with all the fancy DOI’s and what-not that you could ask for. Both the content and the platform itself are licenced under free licences. Reviews are public, either with the authors name or anonymously, which turns out to greatly improve review quality. Reviews are citable objects with DOI’s and everything. A number of reviews get invited, but anyone can submit a review if they’d like. People can post comments on reviews. Their funding comes entirely from sponsors. Their average costs per publication are under $400, way less than the article processing fees of most open access journals. They keep themselves to principles of openness way beyond the Fair Open Access Principles.

Right now SciPost publishes papers in physics. They want to expand to other disciplines, but money is the major bottleneck. Over the past 3 years they’ve gotten around $250k in total funding, so the marginal gains from additional funds should be pretty good.

]]>Continue reading "6. Astronomical waste, astronomical schmaste"

]]>[Part of this badly written blog post has been superseded by a slightly better written forum post over on the EA forum. I might clean up the other parts in the future as well, and if so I’ll publish them at the EA forum as well.]

Previously: [1] [2] [3] [4] [5] … [latest].

*Epistemic status: there is nothing wrong with writing your bottom line first. The purpose of this article is to get my initial thoughts on AI risk down before I start reading more about the topic, because I fear that I might unknowingly grant AI risk proponents that the implicit assumptions they’re making are true. As I procrastinated a lot on writing this post, there have been an number of articles put out that I did not read. I do not intend this document to be a conclusive argument against ai risk so much as an attempt to justify why it might be reasonable to think ai risk is not real.*

Is this text too long? Click here for the summary of the argument.

In this post, I want to tackle the astronomical waste argument as used to justify AI-related existential risk prevention as an EA cause area. I will first describe the argument that people make. After that, I will discuss a number of meta-heuristics to be skeptical of it. Lastly, I want to take the astronomical waste argument face-on and describe why it is so absurdly unlikely for AI risk to be simultaneously real and preventable that the expected value of working on AIS is still not very good.

The astronomical waste argument as most people tell it basically goes like this: the potential good that could be gotten if happy beings colonized the entire universe would be huge, so even if there is a tiny risk of space-colonization not happening, that costs a lot of value in expectation. Moreover, if we can decrease the risk by just a tiny bit, the expected utility generated is still big, so it might be a very cost-effective way to do good.

As many wise people have said before me, “Shut up and calculate.” I will be giving rough estimates without researching them a great lot, because these quantities are not that well-known to humanity either. For the duration of this post, I will be a speciesist and all-around awful person because that simplifies the estimates. Bostrom roughly estimates that colonizing the Virgo supercluster would yield 10^{38} human lives per century. The Virgo SC is one of about 10 million superclusters in the observable universe and we have roughly 10^{9} centuries left before entropy runs out, making a total of roughly 2^{180} potential human lives left in the universe.

I will try to argue that donating \$5000\approx\$2^{13} to an AI risk charity today will counterfactually produce less than one life saved in expectation. To make that happen, we collect 180 bits of unlikeliness for the hypothesis that donating that sum of money to AI Safety organizations saves a lives.

You need to collect less bits if your counterfactual cause area is more cost-effective than malaria prevention. Possibly \log_2(5000/0.20) \approx 14 bits fewer with a charity like ALLFED.

Some of my LessWrong-reading friends would argue that it is impossible to have credence 2^{-200} in anything because my own thinking is fallible and I’ll make mistakes in my reasoning with probability much higher than that. I reject that assertion: if I flip 200 coins then my expected credence for most series of outcomes should inevitably be close to 2^{-200}, because all 2^{200} events are mutually exclusive and their probabilities must sum up to at most 1.

Inhabiting the observable universe might take a really long, and in all this time there is some probability of going extinct for reasons other than AI risk. Hence we should discount the total spoils of the universe by a decent fraction. *30 bits.*

I expect many EA’s to be wrong in their utility calculation, so I think I should propose mechanisms that cause so many EA’s to be wrong. Two such mechanisms are described in previous entries in this series [2] (*9 bits*) [3] (*1 bits*) and I want to describe a third one here.

When we describe how much utility could fit in the universe, our reference class for numbers is “how many X fits in the universe”, where X ranges over things like {atoms, stars, planets}. These numbers are huge, typically expressed as 10^n for n \in \mathbb{N}.

When we describe how likely certain events are, the tempting reference class is “statements of probability”, typically expressed as ab.cdefghij... \%. Writing things this way, it seems absurd to have your number start with more than 10 zeros.

The combination of these vastly different scales together with anchoring being a thing, makes that we should expect people to over-estimate the probability of unlikely effects and hence the expected utility of prevention measures.

I expect myself to be subject to these biases still, so I think it is appropriate to count a number of bits to counteract this bias. *20 bits.*

Nothing is effective in and of itself, effectiveness is relative to a counterfactual action. For this blog post, the counterfactuals will be working on algorithmic fairness and/or digital rights campaigning/legislation, and mainstream machine learning research and engineering. *-1 bit.*

This is a rough sketch of my argument. AI safety can only be an effective cause area if

- The future of the non-extinct universe would be good.
- The probability of an AI-related extinction event is big.
- It is possible to find ways to decrease that probability.
- It is feasible to impose those risk mitigation measures everywhere.
- The AI risk problem won’t be solved by regular commercial and/or academic AI research anyway.
- A single AI-related extinction event could affect any lifeform in the universe ever.
- Without AI first causing a relatively minor (at most country-level) accident first.
- Presently possible AI safety research should be an effective way of decreasing that probability.

I upper bounded the quantity in 1 by 2^{200} good lifes. Properties 2 and 3 are necessary for AI Safety work to be useful. Property 5 is necessary for AI safety work to have meaningful counterfactual impact. Property 6 is necessary because otherwise other happy life forms might fill the universe instead, and the stakes here on earth are nowhere near 2^{200}. If property 7 does not hold, it might mean that people will abandon the AI project, and it would be too easy to debug risky AI’s. Property 8 is in contrast to AI safety work only really be possible after major progress from now has been made in AI capabilities research, and is hence a statement about the present day.

The basic premise of the argument is that there is an inherent tension between properties 2 up to 6 being true at once. AI risk should be big enough for properties 2 and 6 to hold, but small enough for 3 and 5 to hold. I think that this is a pretty narrow window to hit, and which would mean that AI safety is very unlikely to be an effective cause area, or at least it is not so for its potential of saving the universe from becoming paperclips. I am also highly skeptical of both 7 and 8, even assuming that 2 up to 6 hold.

I think it is likely that we won’t be making a what we now think of as “artificial intelligence”, because current conceptions of AI are inherently mystical. Future humans might one day make something that present-day humans would recognize as AI, but the future humans won’t think of it like that. They won’t have made computers think, they would have demystified thinking to the point where they understand what it is. They won’t mystify computers, they will demystify humans. Note that this is a belief about the state of the world, while [2] is about how we think about the world. Hence, I think both deserve to earn bits separately. *5 bits.*

I am not sure that intelligence is a meaningful concept outside principal component analysis. PCA is a statistical technique that gives a largest component of variation in a population independently of whether that axis of variation has an underlying cause. In particular, that might mean that superhuman intelligence cannot exist. That does not preclude thinking at superhuman speeds from existing but would still impy serious bounds on how intelligent an AI can be. *1 bit.*

No matter the above, all reasonably possible computation is restricted to polynomial-time solvable problems, fixed-parameter tractable problems and whatever magic modern ILP-, MINLP-, TSP- and SAT-solvers use. This gives real upper bounds on what even the most perfect imaginable AI could do. The strength of AI would lie in enabling fast and flexible communication and automation, not in solving hard computational problems. I hereby accuse many AI-enthousiasts of forgetting this fact, and will penalize their AI-risk fantasies for it. *2 bits.*

The risks of using optimization algorithms are well-documented and practitioners have a lot of experience in how to handle such software reponsibly. This practical experience literally dates back to the invention of optimization in what is by far my favourite anecdote I’ve ever heard. Optimization practitioners are more responbile than you’d think, and with modern considerations of fairness and adversarial input they’ll only get more responsible over time. If there are things that must be paid attention to for algorithms to give good outcomes, practitioners will know about them. *3 bits.*

People have been using computers to run ever more elaborate optimization algorithms pretty much since the introduction of the computer. ILP-solvers might be among the most sophisticated pieces of software in existence. And they don’t have any problems with reward hacking. Hence, reward hacking is probably only a fringe concern. *3 bits.*

Debugging is a long and arduous process, both for developing software and for designing the input for the software (both the testing input and the real-world inputs). That means that the software will be run on many different inputs and computers before going in production, each an independent trial. So, if software has a tendency to give catastrophically wrong answers, it will probably already do so in an early stage of development. Such bugs probably won’t survive into production, so any accidents are purely virtual or at most on small scales. *5 bits.*

Even if AI would go wrong in a bad way, it has to go really really wrong for it to be an existential thread. Like, one thing that is not ab existential thread is if an AI decided to release poison gas from every possible place in the US. That might kill everyone there, but even the poison gas factories could run indefinitely, the rest of the world could just nuke all of North America long before the whole global atmosphere is poisonous. *10 bits.*

Moreover, for the cosmic endowment to be at risk, an AI catastrophy should impact every lifeform that would ever come to exist in the lightcone. That is a lot of ground to cover in a lot of detail. *10 bits.*

Okay, let’s condition on all the above things going wrong anyway. Is AI-induced x-risk inevitable in such a world? Probably.

- There should be a way of preventing the catastrophies.
*5 bits.* - Humans should be able to discover the necessary knowledge.
*3 bits.* - These countermeasures have to be universally implemented.
*10 bits.* - Even against bad actors and anti-natalist terrorists.
*10 bits.*

Let’s split up the AI safety problem into two distinct subproblems. I don’t know the division in enough detail to give a definition, so I’ll describe them by association. The two categories roughly map onto the distinction from [4], and also roughly onto what LW-sphere folks call the control problem and the alignment problem.

Capitalist’s AI problem | Social democrat’s AI problem |

x/s-risk | Cyberpunk dystopia risk |

Must be solved to make money using AI | Must be solved to have algorithms produce social good |

Making AI optimal | Making algorithms fair |

Solving required for furthering a single entity’s values | Solving required for furthering sentient beings’ collective values. |

Only real if certain implausible assumptions are true | Only real if hedonistic utilitarianism is false, or if bad actors hate hedonistic utility. |

Prevent the light cone from becoming paperclips | Fully Automated Luxury Gay Space Communism |

Specific to AGI | Applies to all algorithms |

Fear of Skynet | Fear of Moloch |

Beating back unknown invaders from mindspace | Beating back unthinkingly optimistic programmers |

Have AI do what we want | Know what we want algorithms to do |

What AIS-focussed EAs care about | What the rest of the world cares about |

I’m calling 20 bits on the capitalist’s problem getting solved by capitalists, and 15 bits on the social democrat’s problem getting solved by the rest of humanity. We’re interested in the minimum of the two. *15 bits.*

There are two separate ways of being inefficient to account for. AIS research might be ineffective right now no matter what because we lack the knowledge to do useful research, or AIS work might in general be less effective than work on for example FAT algorithms.

The first idea is justified from the viewpoint that making AI will mostly involve demystifying the nature of intelligence, versus obtaining the mystical skill of producing intelligence. Moreover it is reasonable to think given that current algorithms are not intelligent. *5 bits.* ((Note that this argument is different from the previous one under the “AI is fake” heading. The previous argument is about the nature of intelligence and whether it permits AI risk existing versus not existing, this argument is about our capability to resolve AI risk now versus later.))

The second idea concerns whether AI safety will be mostly a policy issue or a research issue. If it is mostly policy with just a bit of technical research, it will be more effective to practice getting algorithms regulated in the first place. We can gain practice, knowledge and reputation for example by working on FATML, and I think it likely that this is a better approach at the current moment in time. *5 bits.*

Then the last concern is that AIS research is just AI capabilities research by another name. It might not be exactly true, but the fit is close. *5 bits.*

Let’s get a sense of scale here. You might be familiar with these illustrations. If not, check them out. That tiny bit is the contribution of 4 year researcher-years. One researcher-year costs at least $50,000.

The list of projects getting an ERC Starting Grant of 1.5 million euros. Compared to ambitious projects like “make AI safe”, the ERC recipient’s ambitions are tiny and highly specialised. What’s more, these are grant applications, so they are necessarily an excaggeration of what will actually happen with the money.

It is not a stretch to estimate that it would cost at least $50 million to make AI safe (conditional on all of the above being such that AIS work is necessary). So a donation of $5000 would be at most 0.0001 of the budget. *13 bits.*

I’ll aggregate these because I don’t trust myself to put individual values on each of them.

- Will the universe really get filled by conscious beings?
- Will they be happy?
- Is it better to lead a happy life than to not exist in the first place?
- Is there a moral difference between there being two identical happy universes A and B versus them being identical right up to the point where A’s contents get turned to paperclips but B continues to be happy? And how does anthropic bias factor in to this?
- Has any being ever had a net-positive life?

I listed all objections where I was at least 50% confident in them being obstacles. But there are probably quite a number of potential issues that I haven’t thought of because I don’t expect them to be issues with enough probability. I estimate their collective impact to count for something. *5 bits.*

It turns out I only managed to collect 174 bits, not the 180 bits I aimed for. I see this as weak evidence for AIS being better than malaria prevention but not better than something like ALLFED. Of course, we should keep in mind that all the numbers are made up.

Maybe you disagree with how many bits I handed out in various places, maybe you think I double-counted some bits, or maybe you think that counting bits is inherently fraught and inconsistent. I’d love to hear your thoughts via email at beth@bethzero.com, via Reddit at u/beth-zerowidthspace or at the EA forum at beth.

]]>*I am skeptical of AI Safety (AIS) as an effective cause area, at least in the way AIS is talked about by people in the effective altruism community. However, it is also the cause area that my skills and knowledge are the best fit for contributing, so it seems worthwhile for me to think my opposition to it through.*

Previously: [1] [2] [3] [4] … [latest].

*Epistemic status: this argument has more flaws than I can count. Please don’t take it seriously. [See the post-script]*

Let’s answer this abstract philosophical question using high-dimensional geometry.

I’ll assume for simplicity that there is a single property called intelligence and the only variation is in how much you have of it. So no verbal intelligence vs visiual intelligence, no being better at math and than at languages, the only variation is in how much intelligence we have. Let us call this direction of variation g, scaled to have \|g\| = 1, and pretend that it is roughly the thing you get from a singular value decomposition/principal component analysis of human’s intelligence test results.

A typical neural net has many neurons. For example, VGG-19 has ~143 million parameters. Now suppose that we train a VGG-19 net to classify images. This is an optimization problem in \mathbb{R}^{143 \text{ million}}, and let’s call the optimal parameter setting x. By definition, the trained net has an intelligence of exactly the inner product g^{\mathsf{T}}x. ((Note that the projection of *g* into this 143 million-dimensional space might be much shorter than g itself is, that depends on the architecture of the neural net. If this projection is very short, then every parameter setting of the net is very unintelligent. By the same argument that I’m making in the rest of the post, we should expect the projection to be short, but let’s assume that the projection is long for now.)) ((I’m assuming for simplicity that everything is convex.))

The trained net is intelligent in exactly the extend that intelligence helps you recognize images. If you can recognize images more efficiently by not being intelligent, then the trained net will not be intelligent. But exactly how helpful would intelligence be in recognizing images? I’d guess that a positive amount of intelligence would be better than a negative amount, but other than that I have no clue.

As a good subjective Bayesian, I’ll hence consider the vector \omega of goodness-at-recognizing-images to be chosen uniformly from the unit sphere, conditional on having non-negative intelligence, i.e., uniformly chosen from \{\omega\in\mathbb{S}^{143\text{ million} - 1} : g^{\mathsf{T}}\omega \geq 0\}. For this distribution, what is the expected intelligence \mathbb{E}[g^{\mathsf{T}}x]? Well, we know, we know that x maximizes \omega, so if the set of allowed parameters is nice we would get g^{\mathsf{T}}x \approx g^{\mathsf{T}}\omega \cdot \|x\|, ((I have to point out that this is by far the most unrealistic claim in this post. It is true if x is constrained to lie in a ball, but in other cases it might be arbitrarily far off. It might be true for the phenomenon I describe in the first footnote.)) where \|x\| is how good the net is at recognizing images. We can calculate this expectation and find that, up to a constant factor, \mathbb{E}[g^{\mathsf{T}}\omega] \approx \frac{2}{\sqrt{2e\pi(143\text{ million}-1)}}.

So the trained VGG-19 neural net is roughly 10^{-5} times as intelligent as it is good at recognizing images. Hence, it is probably not very smart.

]]>The standard normal distribution N(0,1) has probability density function \frac{1}{\sqrt{2\pi}}e^{-x^2/2}. There is no way to integrate this function symbolically in a nice way, but we do at times want to (upper) bound expressions of the form \frac{1}{\sqrt{2\pi}}\int_x^\infty e^{-t^2/2} \mathrm{d}t. How can we do this?

One way is to follow this approach. Since t\geq x everywhere, we can upper bound \frac{1}{\sqrt{2\pi}}\int_x^\infty e^{-t^2/2} \mathrm{d}t \leq \frac{1}{\sqrt{2\pi}}\int_x^\infty \frac{t}{x} e^{-t^2/2} \mathrm{d}t = \frac{1}{x\sqrt{2\pi}}e^{-x^2/2}.

There is another tail bound which is a bit weaker for large x, but I like the proof better. We’ll give a tail bound by looking at the moment-generating function \lambda \mapsto \mathbb{E}[e^{\lambda X}], where X \sim N(0,1) is our normally distributed random variable. We can explicitly calculate this expectation and find \mathbb{E}[e^{\lambda X}] = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty e^{\lambda x - x^2/2}\mathrm{d}x = \frac{1}{\sqrt{2\pi}}e^{\lambda^2/2}\int_{-\infty}^\infty e^{-(x-\lambda)^2/2}\mathrm{d}x. The last term is just the entire Gaussian integral shifted a bit and hence \mathbb{E}[e^{\lambda X}] = e^{\lambda^2/2} Now we use Chernoff’s bound (an easy corrollary of Markov’s inequality) to find \mathbb{P}[X \geq t] \leq \mathbb{E}[e^{\lambda X}]e^{-\lambda t}, which we can now minimize over the choice of \lambda, setting \lambda=t, and we conclude that \mathbb{P}[X \geq t] \leq e^{-t^2/2}.

Let X \in \mathbb{R}^d be N(0,I_d) normally distributed, i.e., X is a vector with iid Gaussian N(0,1) entries. What tail bounds do we get on \|X\|? We start off with Markov’s inequality again. \mathbb{P}[\|X\| > t] = \mathbb{P}[e^{\lambda\|X\|^2} > e^{\lambda t^2}] \leq \frac{\mathbb{E}[e^{\lambda\|X\|^2}]}{e^{\lambda t^2}}.

Deriving the moment generating function \lambda \mapsto \mathbb{E}[e^{\lambda\|X\|^2}] of X^2 is an elementary calculation. \int_{-\infty}^\infty e^{\lambda x^2} \cdot e^{-x^2/2} \mathrm{d}x = \int_{-\infty}^\infty e^{\frac{-x^2}{2(\sqrt{1-2/\lambda})^2}}\mathrm{d}x = \frac{\sqrt{2\pi}}{\sqrt{1-2\lambda}}.

The coordinates of X are iid, so \mathbb{E}[e^{\lambda\|X\|^2}] = \mathbb{E}[e^{\lambda X_1^2}]^d = (1-2\lambda)^{-d/2}. The minimizer is at \lambda=(1-1/t^2)/2, and we find, requiring t \geq 1 for the last inequality,\mathbb{P}[\|X\| > t] \leq e^{-d(t^2-2\log t - 1)/2} \leq e^{-d(t-1)^2}.

The operator norm or spectral norm of a n \times n matrix M is defined as \|M\| := \max_{x \in \mathbb{R}^n} \frac{\|Mx\|}{\|x\|}.

Now if M were a matrix with every entry independently N(0,1), what would the largest singular value of this random Gaussian matrix be? I’ll give an easy tail bound based on a *net argument*.

An \eta-net, \eta > 0, on the sphere is a subset N \subset \mathbb{S}^{d-1} such that for every point x \in \mathbb{S}^{d-1} there is a net element n \in N such that \|x-n\| \leq \eta, but every two net elements are at distance at least \eta from each other. A greedy algorithm can construct an \eta-net, and any \eta-net has size at most (4/\eta)^d.

Now let N\subset \mathbb{S}^{d-1} be a 1/2-net. By the above, the size of the net is bounded by |N| \leq 8^d.

The function x \mapsto \|Mx\| is \|M\|-Lipschitz. Hence we can bound \|M\| \leq \max_{x\in\mathbb{S}^{d-1}} \min_{\omega \in N} \|M\omega\| + \|M\|\cdot\|x-\omega\| \leq \max_{x\in\mathbb{S}^{d-1}} \min_{\omega \in N} \|M\omega\| + \|M\|/2. So we have now proved that \|M\| \leq 2\max_{\omega\in N} \|M\omega\|.

Now, as M\omega is N(0,I_d) normally distributed for any \omega\in\mathbb{S}^{d-1}, we can use the union bound over all points of N and conclude that, for all t \geq 1, \mathbb{P}[\|M\| \geq 2t\sqrt{d}] \leq 8^d e^{-d(t-1)^2/2}.

The distribution of the maximum \mathbb{P}[\max_{i \leq n} X_i \geq t] of n independent identically distributed variables X_1,\ldots,X_n \sim N(0,1) is, up to a constant factor, tight with the union bound \mathbb{P}[\max_{i \leq n} X_i \geq t] \leq ne^{-t^2/2}.

Hence the expected maximum is \mathbb{E}[\max_{i \leq n} X_i] = O(\sqrt{\ln n}).

Let x_1,\dots,x_{d+1} \in \mathbb{R}^d be the vertices of a regular simplex such that \|x_i\| = 1 for all i \in [d+1]. If \omega \in \mathbb{S}^{d-1} is chosen uniformly at random, the difference \max_{i,j\in[d+1]} |\omega^{\mathsf{T}}(x_i-x_j| is called the *average width of the simplex*. We can bound this up to a constant factor using our knowledge of Gaussians. Let H_t := \{y\in\mathbb{R}^d : \omega^{\mathsf{T}}y = t\}. The d-2-dimensional volume of H_t\cap \mathbb{S}^{d-1} is (1-t^2)^{(d-1)/2} times the volume of \mathbb{S}^{d-2} by Pythatoras’ theorem. Recalling that (1+1/\lambda)^\lambda \approx e, you can prove that the distribution of \omega^{\mathsf{T}}x_i is approximately N(0,1/\sqrt{d-1}). The Gaussian tail bound now says that the average width of the simplex is O(\frac{\sqrt{\ln d}}{\sqrt d}).

*I am skeptical of AI Safety (AIS) as an effective cause area, at least in the way AIS is talked about by people in the effective altruism community. However, it is also the cause area that my skills and knowledge are the best fit for contributing, so it seems worthwhile for me to think my opposition to it through.*

Previously: [1] [2] [3] … [latest].

There are many people talking about the risks of artificial intelligence. I want to roughly split them into three groups for now, because they worry about very different issues that tend to talk past each other, confusing outsiders.

The **LessWrong-aligned view** seems most popular in the EA community. Examplified by the paperclip maximizer argument, LW-aligned worriers are concerned that an Artifical General Intelligence (AGI) would accomplish their objective in unforeseen ways, and as a consequence should be treated like you should treat an evil genie, except it’d be worse because it would have less understanding of basic words than philosophers have. The principles that AI should satisfy are listed by the Future of Humanity Institute. [Though I suspect at least some of the signatories to have the FATML-aligned view in mind.] A popular book on this is Superintelligence by Nick Bostrom.

**Fairness, Accountability and Transparency in Machine-Learning** (FATML) is a subfield of machine learning, concerned with making algorithmic decision making fair, accountable and transparent. Exemplified by Amazon’s recent recruiting debacle, FATML-aligned worries are concerned that modern algorithmic decisionmaking will exacerbate existing social, economic and legal inequalities. The princples that AI should satisfy are listed by The Public Voice, and these Google ML guidelines fit as well. [Though I suspect at least some of the signatories to have the LW-aligned view in mind.] Popular books include Weapons of Math Destruction by Cathy O’Neil, Algorithms of Oppression by Safiya Noble and Automating Inequality by Virginia Eubanks.

**Other AI-related worries** commonly heard in the media, that I want to separate from the previous two categories because, compared to the above categories, these issues are more about politics and less of a technical problem. Worries include killer drones, people losing their jobs because AI replaced them, and who the self-driving car should run over given the choice.

In the next couple of posts on AI-related topics, I will focus on the first two categories. My aim is to use the FATML-aligned view to compare and contrast the LW-aligned view, hopefully gaining some insight in the process. The reason I separate the views this way, is because I agree with the FATML-aligned worries and disagree with the LW-aligned worries.

]]>[This badly written blog post has been superseded by a slightly better written forum post over on the EA forum.]

*I am skeptical of AI Safety (AIS) as an effective cause area, at least in the way AIS is talked about by people in the effective altruism community. However, it is also the cause area that my skills and knowledge are the best fit for contributing, so it seems worthwhile for me to think my opposition to it through.*

Previously: [1] [2] … [latest].

My background makes me prone to overrate how important AI Safety is.

My fields of expertise and enjoyment are mathematics and computer science. These skills are useful for the economy and in high demand. The general public is in awe of mathematics and thinks highly of anyone who can do it well. Computer science is the closest thing we have to literal magic.

Wealth, fun, respect, power. The only thing left to desire is cosmic significance, which is exactly the sales pitch of the astronomical waste argument. It would be nice if AI-related existential risk were real, for my labour to potentially make the difference between a meaningless lifeless universe or a universe filled with happyness. It would give objective significance to my life in a way that only religion would otherwise be able to.

This is fertile ground for motivated reasoning, so it is good to be skeptical of any impulse to think AIS is as good as it is claimed to be in cost-effectiveness estimates.

]]>[This badly written blog post has been superseded by a slightly better written forum post over on the EA forum.]

All sentences are wrong, but some are useful. I think that a certain emotional salience makes us talk about AI in a way that is more wrong than necessary.

A self-driving car and a pre-driven car are the same thing, but I can feel myself thinking about the two in completely different ways.

Self-driving cars are easy to imagine: they are autonomous and you can trust the car like you trust cab drivers; they can make mistakes but probably have good intent, when they encounters an unfamiliar situation they can think about the correct way to proceed, and if something goes wrong then the car is at fault.

A pre-driven car are hard to imagine: it has to have a bunch of rules coded into it by the manufacturer and you can trust the car like you trust a bridge; it does exactly what it was built to do, but if it was built without proper testing or calculations, things will at some point go wrong. When it does, the company and engineers are at fault.

You can make these substitutions on any sentence in which a computer is ascribed agency. In the best case, “The neural network learned to recognize objects in images” becomes “The fitted model classifies images in close correspondence with the human-given labels”. In reality, that description might be too generous.

It helps to keep in mind the human component. “The YouTube algorithm shows you exactly those videos that make you spend more time on the platform” is accurate in some sense, but it completely glances over the ways in which in the algorithm does not do that. When you listen to music using YouTube’s autoplay, it isn’t hard to notice that suggestions tend to point backwards in time compared to the upload date of the video you’re watching right now, and that, apart from preventing repeats, autoplay is pretty Markovian (that is mathspeak for the algorithm not doing anything clever based on your viewing history, just “this video is best followed by that video”). Both of those properties are clearly a result from the way in which YouTube’s engineers modelled the problem they were trying to solve, I would describe YouTube’s suggestion as “The YouTube autoplay algorithm was made to link you to videos that most people watched and liked after watching the current video”.

When you rewrite AI-related statements, they tend to become more wordy. That is exactly what you would expect, but does make it unwieldy to have accurate conversations. I leave the search for catchy-but-more-accurate buzzwords as an open problem. I am particularly interested in how to translate the term “artificial general intelligence” (AGI).

]]>Sometimes you hear a word or concept that changes how you look at the world. For me, these include *speciecism* and *epistemic injustice*.

**Speciecism** is analogous to racism and sexism, but for species: treating another being differently because they are of another species. Speciecism is about intent; if you eat chickens because they are chickens and not humans, that is speciecist, but if you eat chickens because you concluded from observation that they are incapable of suffering, that is not speciecist.

**Epistemic injustice** is when someone is wronged in their capacity as a knower. If you unjustly limit somebody’s ability to access or express knowledge, like forbidding them from learning to read or speak, that is an epistemic injustice.

I am an outspoken anti-speciecist and I think we should do what we can to prevent epistemic injustice in all forms. But some animals have learned enough language to meaningfully communicate with humans. Does that mean I should find it reprehensible that there are no schools for animals? I think I should and I think I do, but I feel hesitant to firmly claim the position.

]]>I recently picked up programming again. I used to do it a lot before I went to university, but the constant mind-numbing programming assignments quickly put me off of programming. Apart from the occasional quick bug fix for software I use myself, I haven’t done any serious coding for years.

Until recently, when I needed something coded up during my research. I decided to learn Python, and I like it. It is easy to use, the libraries are extensive and user-friendly, and ipython is a useful tool. There is just one thing that draws my ire: the weak type system. Studying math has given me an appreciation for type checking that is even stricter than most languages.

An example: my length in centimeters plus the outside temperature in °C right now equals 180. This calculation makes no sense, because the *units don’t match*: you can’t add centimeters to degrees Celcius. But then there’s Python, which just lets you do that.

`In [`

`1`

`]: length = 170`

In [`2`

`]: temperature = 10`

In [`3`

`]: length + temperature`

Out[`3`

`]: 180`

Most bugs that stem from typos are of this sort. Those bugs are possible because the type system is too weak. If you have two loops, one iterating over `i`

and one over `j`

, basic unit-based type checking would probably flag any instance of `i`

in a place where you should have typed `j`

instead. If you intend to query `A[i][j]`

then it should be possible to let `i`

have row-index type and `j`

have type-index type, making `A[j][i]`

raise a type error.

Another example: Let A \in \mathbb{R}^{n \times n}, x \in \mathbb{R}^n, and we’re interested in the quantity Ax \in \mathbb{R}^n. If you’re like me and you can’t remember what rows and what columns are, then that doesn’t have to impact your ability to symbolically do linear algebra: the quantities xA = A^{\mathsf{T}}x, Ax^{\mathsf{T}} and A^{-1} x don’t “compile”, so any mathematician that reads it will know you typo-ed if you wrote one of those latter expressions. All operations might be matrices acting on vectors, but the matrices A^{-1} and A^{\mathsf{T}} fundamentally take input from different copies of \mathbb{R}^n than the ones that x and x^{\mathsf{T}} live in. That is why matrix operations make sense even if the matrices aren’t square or symmetric: there is only one way to make sense of any operation. Even if you write it wrong in a proof, most people can see what the typo is. But then there’s Python.

In [4]: import numpy as np

In [5]: x = np.array([1,2])

In [6]: A = np.array([[3,4],[5,6]])

In [7]: np.dot(A,x)

Out[7]: array([11, 17])

In [8]: np.dot(A,np.transpose(x))

Out[8]: array([11, 17])

In [9]: np.dot(x,A)

Out[9]: array([13, 16])

I am like me and I can’t remember what rows and what columns are. I would like the interpreter to tell me the correct way of doing my linear algebra. At least one of the above matrix-vector-products should throw a type error. Considering the history of type systems, it is not surprising that the first languages didn’t introduce unit-based types. Nonetheless, it is a complete mystery to me why modern languages don’t type this way.

]]>*TLDR: if welfare compounds then risk-aversion is good.*

Within EA circles, the question of splitting donations pops up every once in a while. Should you donate all your money to the singular top-rated charity your singular top-rated cause area, or is there reason to split your donations between various different causes or interventions?

People other than me have written and talked about this under various headers, I’ll list a small subset. Reasons not to diversify (Giving What We Can). Reasons to diversify: the value of information, explore vs exploit (Amanda Askell @ 80k). Reasons both for and against: risk aversion, diminishing returns, EV maximization (Slate Star Codex). In-depth blog post with mahy arguments both for and against (EA forum). Not listed but probably talked about before: splitting your donations gives you extra practice at donating which might lead to you making better donation decisions in the future.

In this post I want to make an argument in favour of splitting donations based on compounding economic returns and measurement error. Specifically, compounding returns favour more consistent growth over a slightly higher but variable growth.

Let’s consider a 100-year time horizon. Suppose that there are 100 charities, C_1,\dots,C_{100}, whose effectiveness is heavily-tailed: donating $1000 to charity C_i allows them to produce i*\$1000 in welfare after a year. Charity evaluator BestowCapably measures the effectiveness of every charity C_i every year j and finds an effectiveness of i + s_{i,j}, where the s_{i,j} are independently normally N(0, \sigma^2) distribution. Let’s assume BestowCapably’s measurement error \sigma does not go down over time.

The way I think of these quantities is that effectiveness is a heavy-tailed distribution and that measurement error is multiplicative (instead of additive).

We assume all welfare gains are reinvested in charity the next year, so that the gains compound over years. The initial welfare is 1. We consider three different donation strategies: donate everything to the single best rated charity, split the donation between the top three rated charities, or split the donation between the top ten rated charities. We plot the compounded welfare after 100 years versus \sigma below.

In the above graph, we see that,for low measurement error, donation splitting is worse than donating everything to the best charity, but for high measurement error, the situation reverses and splitting donations wins out.

The code I’ve used (included below) to simulate the scenario has a couple *researcher degrees of freedom*. It is unclear whether measurement error should scale with charity effectiveness. I used Gaussian noise without any justification. My choice of range of \sigma to plot was chosen to have a nice result. The range of charity effecicies has close to no justification. The same stable result can be gotten by donating everything to AMF and nothing to speculative cause areas. The splitting incentive I illustrated only holds at the margin, not for the average donation. Because \sigma is fixed, the magnitude of the effect of donation splitting in this model depends heavily on the number of charities (less charities means greater effect).

Nonetheless, if you care about multi-year impacts, it might be wise to consider more than just short-term expected value. Risk-aversion translates to expected counterfactual impact when results compound.

```
import random
import matplotlib.pyplot as plt
import math
charitycount = 100
yearstocompound = 100
# The charities are {1,...,n}
# Charity i has effectiveness i
# Effectiveness measurement carries exp noise of size stddev
# Outputs list of (i, i + noise)
def measurecharities(n, stddev):
charities = []
for effectiveness in range(1,n+1):
charities.append((effectiveness,random.gauss(effectiveness,stddev)))
return charities
# Given list of tuples (x, y),
# calculates the average of x's for
# the k tuples with highest y value.
def avgtop(list, k):
sortedlist = sorted(list, key=lambda tup: tup[1], reverse=True)
sum = 0.0
for i in range(k):
sum += sortedlist[i][0]
return sum/k
# Split donations among k charities
for k in [1,3,10]:
x = []
y = []
# We plot the effect for different noise magnitudes
for stddev in range(1,251):
logwelfare = 0.0
for i in range(yearstocompound):
welfaregain = avgtop(measurecharities(charitycount, stddev), k)
logwelfare += math.log(welfaregain)
x.append(stddev)
y.append(max(1,logwelfare))
plt.plot(x, y,label=k)
plt.legend()
plt.xlabel('Error in measuring effectiveness')
plt.ylabel('Log(' + str(yearstocompound) + '-year compounded welfare gains)')
plt.title('Donating to top k out of ' + str(charitycount) + ' charities')
plt.show()
```

]]>Not (best (things of 2018)) but ((best things) of 2018), because recommendations get more interesting if they are rate-limited and less interesting if a recency constraint is imposed.

By internet creators Vi Hart and Nicky Case; Parable of the polygons. Cute little triangles and squares get segregated in ways none of them ever intended against their best wishes.

Portraying one of the most important trans people of the past few years, Vice Broadly’s piece on Caitlyn Jenner was a nice read.

On why setting maximum prices is bad. They Clapped by Michael Munger. Very salient, go read it.

I see a lot of talks from computer science researchers, and CS people are surprisingly good at giving captivating talks. But, quoting Virginia Woolf,

[..] one must read [any book] as if it were the last volume in a fairly long series, continuing all those other books that I have been glancing at. For books continue each other, in spite of our habit of judging them separately.

Virginia Woolf, A Room of One’s Own, or page 52 in Penguin’s Vintage Mini “Liberty”

And so a talk must be considered in its social context. Based on this principle, the clear winner for this category is this keynote speech by James Mickens of Harvard University at USENIX Security 2018: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? Mickens is a captivating orator, the talk is funny and informative and gives a critical view on an important issue of the present day.

An old one for nostalgia. How to spot photo manipulation. Body By Victoria. Do click the links to follow-up posts, and the rest of the website is worth checking out as well.

This text fragment reflects every interaction I’ve had with psychologists anywhere, both my gatekeepers and psychologists I visited for other reasons.

My anorexic patients sometimes complain of being forced into this mold. They’ll try to go to therapy for their inability to eat a reasonable amount of food, and their therapist will want to spend the whole time talking about their body image issues. When they complain they don’t really have body image issues, they’ll get accused of repressing it. Eventually they’ll just say “Yeah, whatever, I secretly wanted to be a ballerina” in order to make the therapist shut up and get to the part where maybe treatment happens.

Scott Alexander, Del Giudice On The Self-Starvation Cycle

This is not really a contest, Contrapoints’ The Aesthetic is the most beautiful piece of film I’ve seen in years. It is an honest expression of feelings and internal dialogue and conflict that trans women experience. It touches on so many uncomfortable issues without having any single clear message. Contrapoints raises the video essay to form of art. There is so much going on so many levels and I can just keep on watching the thing over and over again. Highly recommended watching for both trans and cis people.

The creator got quite some social media backlash on the video. There is exactly one reaction video that I felt was worth watching. Nobody Wins: ContraPoints, The Aesthetic, and Negative Representation by let’s talk about stuff. [This text essay is also pretty good. How Contrapoints Misunderstands Gender.]

My choice of best book for 2018 is Aphro-ism by Aph Ko and Syl Ko. It is a blog-turned-book, with a number of brilliant essays on, among others, veganism and social justice. I cannot overstate how much I like this book. I learned a lot from reading this book, and not just about the book’s subject matter.

The writings of the Ko sisters are very far from every thought I’ve ever had. This fact is reflected in how much I learned from the book, as well as in how difficult it was to understand it. I’ve re-listened this book 5 times by now. The first time, I understood literally nothing. Each time after that I understood a bit more, and I feel I understand most parts now. Not yet at the level of being to explain the ideas, but at the level of seeing good use value in them.

]]>What is better, if everyone is wrong about the same 2% of facts, or if everyone is wrong about a different 4% of facts? Depending on how you answer this question, you should act in very different ways. I’ll take vegan advocacy as an example, but the question can be applies more generally.

If you’re in the first group, you would prefer a scientific data-driven approach. You would experiment with many different approaches to advocacy, analyse the data to find the single best way of doing outreach, and make everyone in vegan activism aware that this is the best way to do it.

If you prefer the 4% case, a local algorithm is the way to go. Think about what drove you to become vegan, and continue this strategy. If you were shocked into becoming vegan by a Cube of Truth, you should be participating in Cubes of Truth. If you became vegan after your friendly vegan neighbour exemplified that veganism is a totally normal lifestyle and they allowed you to pick their brain about why they became vegan themselves, then you should become the friendly vegan acquaintance of the people you know yourself.

One interesting question if you enact the local algorithm, is how to weigh anecdotes. The local algorithm described above only considers your data; one alternative algorithm is to use the approach that was effective on the majority of your direct friends that became vegan before you. Another algorithm looks at all your the friends of your friends, or everyone within distance 3 in the friendship-graph. If everyone is connected by everyone by a friendship-path of length 6, then the distance 6 algorithm is exactly the data-driven approach from the second paragraph.

Evolutionary theory suggests that the small-distance algorithms are effective, for the best outreach strategy will eventually out-compete all others. But for the distance 0 or 1 cases, you’re basically working on anecdotal evidence. I’m not sure anymore what the correct value is to place on anecdotes.

]]>Suppose there was a gene, let’s call it gene X, that predisposes you never to participate in any study as a subject. It makes you decline all requests to record your data for research purposes and refuse to ever vote for anything.

Carriers of gene X might be at increased risk for cancer. They might be invulnerable to hemlock. They might all individually have an IQ of exactly 120.2. Maybe they never forget where they left their keys, or grow an extra belly button during their third puberty. Possibly they’re all called Alex and stop aging when 50 years old, only to suddenly die on the day they turn 90 years old. Hell, they might be 20% of the world population, favour the Libertarian Party and think Estonia should have won the last Eurovision song contest. We’d never know because nobody can study them.

On the other hand: suppose there is some subpopulation that always participates in studies, and forms a big fraction of participants in nearly any study. It makes sense to want to be a part of this subpopulation, because that way every study ever will be more descriptive of you. Luckily, you can become part of this group simply by committing to participate in every study you are asked for from now on. Except that reading this post might have caused you to make that commitment while you would never have done so yourself. You can now only become part of a new subpopulation, the one that commits to being studied after reading a participation bias-based suggestion to do so.

]]>When the GDRP became effective, Tumblr decided to break its RSS feeds for all EU residents. When you try to fetch `https://username.tumblr.com/rss`

, they’ll serve their GDRP wall instead of your requested file. You can only grab the feed if you possess the correct cookies.

Anyway, here is a hacky fix for your RSS feeds. I’m assuming you possess an http server yourself. I use a Raspberry Pi with lighttpd and selfoss. I’m assuming your user on the server is called `beth`

, you want to follow the user called `username`

on Tumblr and your document root is `/home/beth/public`

.

Create the folder `/home/beth/public/rss`

. Create the file `/home/beth/.bin/fetchfeeds.sh`

with the following contents. Duplicate the last line once for every user you want to follow, and adjust all three occurences of username to fit.

```
#!/bin/bash
curl --header 'Host: username.tumblr.com' --header 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/71.0.3578.80 Chrome/71.0.3578.80 Safari/537.36' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8' --header 'Accept-Language: nl,en-GB;q=0.9,en;q=0.8' --header 'Cookie: rxx=1jo2mpfhsia.1c2ecvc5&v=1; pfg=1bc46aba34ffeb83e2ef0859447d282cf8a8a2a9f95200a2a705f3afebfe9bef%23%7B%22eu_resident%22%3A1%2C%22gdpr_is_acceptable_age%22%3A1%2C%22gdpr_consent_core%22%3A1%2C%22gdpr_consent_first_party_ads%22%3A1%2C%22gdpr_consent_third_party_ads%22%3A1%2C%22gdpr_consent_search_history%22%3A1%2C%22exp%22%3A1576795974%2C%22vc%22%3A%22%22%7D%237343812268; tmgioct=5c1acbc67c0f570418402840' --header 'Connection: keep-alive' 'https://username.tumblr.com/rss' -o '/home/beth/public/rss/username-tumblr.rss' -L
```

[The curl command was produced using the CurlWget browser extension.]

Don’t forget to fix the permissions: `chmod +x ~/.bin/fetchfeeds.sh`

. Now put this in your cron table by using the command `crontab -e`

: `0 * * * * ~/.bin/fetchfeeds.sh`

. This will execute the bash file in the first minute of every hour.

Lastly, put `http://localhost/rss/username-tumblr.rss`

in your RSS reader.

Context: Butarin on non-financial applications of blockchain, 15 tweets. I’ll assume you have read it.

I am deeply convinced that blockchains have no use cases. Trust is provided in the real world by the option to sue people. Trust in most blockchains is misplaced because miner pools are too big and the group of dictators of any coin is small, not accountable and wrongly incentivised. Timestamping data works just as well without using a distributed blockchain. Immutability is provided by standard crypto. The oracle problem is a real fuckin’ serious problem that is only ever resolved through the force of law.

Every function of distributed blockchains can be provided by combining digital signatures, cryptographic commitment schemes, public key cryptography, hashing all data and either periodic or on-demand polling.

So every time Tyler Cowen of Marginal Revolution posts a link to Buterin’s Twitter feed, my confidence in other people’s assertions dies a little bit more.

I’ll take the tweets linked up top in order.

2-5. I think he means “Cryptography allows you to encrypt data, prove data was signed by someone, etc etc… blockchains OTOH allow you to prove that a piece of data was *not* published [according to protocol]”, because the original is obviously false (my daily newspaper published things but is not on any blockchain). The adapted statement is also false, because SSL is based on the very notion that certificates can be rejected and that you can check that a certificate hasn’t been rejected.

6. Not just SSL, GNuPG does this very same thing as well.

7-8. Blockchains aren’t credibly neutral, every miner on a blockchain has money in it, skewing their incentives in certain directions. Trust only goes as far as you can pay. Cryptographic hashes and signatures can make every database trusted if you can sue its proprietor.

9-11. This might be true, but it is pretty much impossible that performance will ever reach the one of standard crypto, and Buterin conveniently manages to forget the additional cost in programmer-hours. Technological cost is human cost, of the kind which is in most limited supply.

12. This tweet illuminates a lot. Buterin lives in countries without proper online banking and moreover hates privacy of any kind.

13-14. What does this even mean. Anything can and will get hacked. Your hard drive *will* stop working one day, and your backup won’t work.

15. Smaller stakes \implies smaller mining rewards \implies less miners \implies easier to attack. Also applications breaking incurs a huge social cost; some people are still distrustful of others from when Google Reader (RIP) was cancelled.

I know a lot of computer scientists at my institute, including people who do research into applications of blockchains. Nobody I know believes blockchains are useful apart from black markets, tax evasion, as a novel pyramid scheme, or as a hype for getting grant money.

I get why Cowen falls for Buterin’s sweet talk, he is an economist with no knowledge of cryptography. I don’t get why not more people who know anything about cryptography are speaking out about the insanity of blockchain. My best guess is that everyone who knows stuff about it is using it to either make money themselves, pretending to be enthusiastic to further the pyramid scheme, or profiling themselves as “blockchain expert” and charging high consultancy fees.

I hope I’m completely wrong and I just misunderstood every statement every blockchain fanatic has ever made. My constant rejection of all their statements does make me question myself, because it seems strange that so many people can be so consistently wrong about a thing. I’d love to show epistemic humility here, but I don’t consider it to be justified.

]]>