SoundLogic

reading the crafting rules in pathfinder 2e and like. yeah i realise the system is not meant to model a functioning economy with supply and demand, but since the thing it does seem meant to do is 'make sure crafting isn't more than marginally useful or it'll break the game' i resent it a little

@transunlight

at least for magic items they seem strictly worse (at least re: making sense of why anyone spends their time crafting high-level ones for resale). the ordinary crafting ones maybe are better, idk

Well yes, but they're extremely good at their design goals, which is to explain why no one spends their time crafting magic items for resale.

Fairly explicit in the rules that you can't actually buy magic items, and that crafting an item is a quest, and for the most part you're supposed to use whichever things you find.

(3E had a major change when it fairly explicitly allowed purchase of magic items, rather than explicitly forbidding it; this led to great gnashing of teeth. It was a big part of the shift to planned builds, away from most of the your character being determined semi-randomly.)

based on your tags and also the words "3E" you are talking about D&D, and for all I know what you are saying is true in early editions of D&D but this post is about pathfinder, which totally does allow you to buy magic items from merchants. said merchants must be either making them for resale themselves or buying them from someone who is.

Oh yeah, I misread you when I saw 1E, sorry; I forgot there was a pathfinder 2E.

(Pathfinder 1E is essentially D&D 3E, with minor tweaks. And I think you're right there; the rules explicitly let you buy items, and they make it predictable and straightforward to make magic items, but from my memory it's not a very reasonable income source and also you make the same income/day from low-tier items that you do from high-tier items.)

i mean trying to pretend that the mechanics are simulationist or that NPCs all function according to the same rules as PCs has always felt like catastrophically missing the point to me?

In most of the games I'm familiar with (which I know is a specific subset of ttrpgs) you're very much supposed to be treating mechanics that way.

Like "making magic items is hard by the rules, and that's why you can't buy them in shops" isn't me making an inference; I'm pretty sure that's literally what the DMG said. And simulationist verisimilitude was an explicit goal of 3e, and they did a ton of fairly impressive math work to make it work smoothly. (There's a fun essay somewhere working through how an optimized level 5 expert puts up nearly exactly world-record numbers in multiple track and field sports.)

In practice I'm not really a ttrpg person at all; but I would have absolutely zero interest in a game that didn't at least try to make the NPCs follow the same rules as the PCs. Otherwise how are you supposed to understand how the world works?

by reading worldbuilding fluff and incorporating your existing knowledge and intuitions, same as every other fantasy setting such as those in novels and video games

I think this is related to the thing where I just don't get magical realism.

But my reaction to that is, yes obviously, but my knowledge and intuitions have to be adapted to the rules of the setting: if some poeple can fly, or read minds, or reliably survive two-hundred-foot falls, or murder a thousand ordinary men with a teaspoon, then I have to use my knowledge and intuitions to think about how a world with those abilities would work.

And so I'd say the same thing about worldbuilding fluff: it's useful, but it has to be compatible with what you tell me about how things work mechanically. If powerful wizards have the ability to mind-control millions of people, then I'll ask why any nations aren't run by mind controllers; if powerful wizards have the ability to create eighteen tons of worked iron for fifty gold then I'll ask why there are any powerful wizards willing to work for cash if iron is worth money.

(Pathfinder d20 "solves" this problem by including a clause that "this iron is "not suitable for use in the creation of other objects and cannot be sold", which is the sort of thing that genuinely offends me because that's not how iron works and now I can't use my knowledge and intuitions.)

To use intuitions, to have worldbuilding, you need some sense of what people are capable of doing. And that's what the rules are there for.

I don't think the magical-realism analogy works very well here. There's an abstraction-layer you're missing, I think, of the difference between player-facing options and character-facing options, which isn't replicated in magical realism.

In magical realism—or, at least, all but the most aggressively metafictional magical realism—the thing that's happening isn't "the magic happens only in the narration, not in-universe, while in-universe events are meanwhile fully coherent and consistent-with-ordinary-human-psychology but look very different from what's narrated"; but, in many TTRPGs, that's precisely the thing going on. This varies in how overt it is, but it's in fact all over the place.

For a very overt example of this, see e.g. Gone to Hell, in which players get access to such actions as "attempt a clever solution that just makes the problem worse" and "fail to anticipate the obvious consequences". Clearly, this isn't the player character choosing to do these things; rather, it's the player choosing for the player character to do these things, causing the player character, in-story, to do them unintentionally while attempting to do other things. There's going to be a coherent in-universe account of why the PC attempted their clever solution, and of why it ended up backfiring; that account isn't going to look at all like "they decided to take the action of 'attempt a clever solution that just makes the problem worse'".

For a less overt example, meanwhile, I'd point you in directions like the D&D 3.5 / Pathfinder 1 barbarian's Rage ability, whose initiation is entirely under player control, even as in-universe the thing it represents is typically a much less controlled sort of rage; it would be a decidedly unconventional flavoring for barbarians, for them to be people of very exceptionally strong mental self-control who deliberately choose to enter the enraged mental state for precisely only those moments when their doing so appears tactically-ideal in combat, even as that's precisely what their player is likely to be doing most of the time. And, much like the Gone to Hell case, this is implicitly justified by in-universe causal pathways different from the player-facing ones, with in-universe events happening to push the character into / out of the relevant psychological state with timing concurrent to when the player chooses to enter / exit the relevant game-mechanical state.

The thing rules like the ones in these two examples are doing, then, isn't describing what options the player characters are choosing to take. Rather, what they're doing is offering the players levers of influence over the direction in which the player characters' story goes. Unlike in magical realism, when you look underneath, there's still room for a fully consistent causal story about what happened in-universe, much as a well-told (non-magical-realism) fantasy novel will typically have a causal story about why in-universe goings-on are going on beyond "the author decided to have these things happen".

And then, once you have rules like that, it's easy to see how worldbuilding comes in as a thing that can exist separately from, but coexist with, them: the worldbuilding defines what the in-universe causal pathways are by which the player-selected outcomes end up taking place. The player chooses to have their character fail to anticipate an obvious consequence to an action; the worldbuilding supplies the background-information that cognition-impairing magic exists and could have been cast on the player's character in such a way as to cause that failure-of-anticipation.

(The thing with the iron, I agree, is silly, and I will make no attempt to defend that one. That's a case where multiple systems which are trying to describe principles of how things work in-universe are in conflict with one another around the edges, without any of the involved systems being of the purely-player-facing variety.)

See, I think some of that is exactly what I meant by drawing the magical realism comparison. In the mechanisms you describe, things are happening for narrative-logic reasons rather than diegetic reasons, which is the same thing I'm objecting to in magical realism.

Like, in your Gone to Hell examples, it seems like the reason your solution makes the problem worse is that narrative fiat says it makes the problem worse, because you took the narrative action "try to help and make the problem worse". I do take your point that you're then narrating in-universe reasons that works, but that's sort of fundamentally unsastifying to me.

(This does overlap with my distaste for narrative games in general, and honestly my discomfort with roleplaying; I'm me, and I'm going to take the actions that I think best achieve my goals. "Try to help and make the situation worse by accident" is not going to achieve my goals so it's not an action I would choose to take.)

In contrast, your barbarian example I think precisely carves out where we differ: I feel like you have to interpret Barbarian Rage as a conscious choice that barbarian characters are making, because that's what the rules give you and the world functions according to rules where barbarians choose to rage or not for tactical reasons. Now you could always choose to roleplay a character who rages at inadvisable times, but the rules to me are very clearly spelling out barbarians who choose to rage on purpose.

To try to focus in a bit on some of my real issue here: in the (very good) webnovel Practical Guide to Evil, there are narrative causal pathways where things are actually more likely to happen when they advance a character's story, or something like that. And the characters in the world know that and they exploit it to make predictions and build their tactics; they say things like "Ah yes, but this is the beat in the story where the villain gets beaten but escapes and lives to fight another day" and then they rely on that to plan their next move because it's a reliable part of the universe's physics.

If your rules cause things to happen for narrative reasons, you live in a universe that has narrative causality. And in that case it's irrational, and a little bit insane, to not incorporate that narrative causality itself into your model of the world and let it influence your next decisions.

I think some important piece of my disconnect from you, here, is that I don't understand why you're analogizing TTRPGs with player-facing rules which shape events in accordance with narrative causality to APGTE—where narrative causality is an in-universe phenomenon—as opposed to analogizing them to more ordinary fantasy novels like, to pick an example I'm pretty sure we've both read, the Stormlight Archive.

It is not the case, on Roshar, that the world runs according to narrative causality. That's not part of the setting. This is true irrespective of the fact that the stories of Roshar to which we have access have been written by Brandon Sanderson and thus are in fact shaped in accordance with narrative causality. It isn't insane for the characters on Roshar to refrain from thinking about their lives in narrative terms; by the rules of their universe, this would not produce good predictions most of the time, even if it's the case for us as readers that making predictions based on narrative causality can be effective when reading novels set on Roshar.

As far as I can tell, the sorts of tabletop RPGs where the game rules don't represent the in-universe rules are pretty directly analogous to that, most of the time. I'm sure there are some which are more APGTE-like, where player-facing narrative-outcome-selection is an in-universe phenomenon; but that's nondefault. It seems to me that it's about as much of a misinterpretation of most games-with-rules-that-aren't-strictly-modeling-in-universe-causality to claim that their rules imply that their worlds run on narrative causality as an in-universe phenomenon as it would be a misinterpretation of Brandon Sanderson's novels to say that their worlds run on narrative causality as an in-universe phenomenon. (Which most of them don't! Perfect State is an outlier.)

I think I'm claiming that if the rules encode narrative causality then the world-as-the-player-experiences it kind of has to have narrative causality. From the perspective of the player character, you can in fact make better predictions and decisions if you take into account the structure of the player-facing narrative-outcome-selection.

And the only reasonable response for those characters to make is to, like, try-to-help-but-make-things-worse at the villain whom they want to fail, or something like that.

And I realize that I'm fighting the system here, in that some of these narrative games are asking you to take the role of the author rather than the character in some sense. But even if I buy that, I feel like the character should be taking the role of the character, and if the rules cause things to happen for narrative reasons then the character is living in a world where things sometimes happen for narrative reasons, and this seems like something they should be able to figure out and exploit that.

And I guess your point is, in most novels the characters live in worlds where things happen for narrative reasons and they don't take that into account. But honestly I think in most of the fiction I read they do, but only a little; they go on and on about the prophecy or the will of fate or the way things are meant to fall out or whatever. Knights Radiant get an explicit powerup in the scene where they come to climaxes in their character arcs! (see also "The Well-Tempered Plot Device".)

But there's also definitely an element of, if I'm playing a character, I want to inhabit the character, and make good decisions that will make them happy.

So, I run a campaign and the NPCs follow 'different rules' than the PCs. But they are the same physics.

Specifically, I model it as different 'packages' of abilities that people are taking.

Vanilla as written, someone can hold the breath for a number of rounds equal to twice their constitution score, reduced by certain activities.

In my handling, the typical civilian can hold their breath for a number of rounds usually from half their constitution score to twice their constitution score, and a big part of this is that they are implicitly doing 'actions' that reduce that.

An adventurer, or other people with relevant training, get twice their constitution score approximately always, and reduce the space of activities that cut down on the time to almost only 'productive' activities.

The PCs are the sort of people who can do things like by pure force of their own soul un-dislocate their own shoulder. In universe, this is the same sort of process that everyone uses to be able to move at all.

The PCs activity patterns have had downsides. While their access to magical healing prevents them from many of the ailments that the average peasant may suffer, they have exotic problems caused by massive magic channeling, or having their souls almost yanked out in combat, and all that sort of crazy shenanigans.

But also, they aren't as good as an equal level civilian librarian at being a librarian. They traded in some of the skills such would develop for things like "the ability to keep fighting until zero hitpoints without freaking out or collapsing, and then even at zero HP carefully move to only finally pass out when they have chosen to strain themself further".

In fact, for a side adventure the players made PCs with 'NPC' builds! I haven't stated out a lot of the implicit NPC build stuff in as much detail as I have for the stuff the PCs use, but I also haven't stated out exotic monk abilities nearly as much.

It doesn't have to be 'narrative rules' vs 'IC rules'. The rules already create divides that shouldn't strictly exist IC. Not just between PC and NPC, but between ranged and melee weapons.

If you attach a string to an arrow in a bow and arrow and tie it to yourself, it is still a ranged weapon. But one can in principle draw a sequence of individually small changes that eventually starts with a bow and arrow and ends with a longsword, and attacking with those uses different rules!

How do we reconcile this?

My answer is: A lot of those intermediate weapons are going to be awful and using them would be dumb. IC, there is actually a smooth gradient. It involves adding a lot of penalties to your attack roll, then removing a lot of penalties from your attack roll, until you wind up somewhere else. I have not stated it out in detail because it will not come up on screen to the point of needing much detail.

What are the rules for being a civilian sustenance farmer? What perks do you get as you level up? Probably some. Also haven't stated out in detail.

And what are the exact rules for being not an adventurer with a tendency to craft magic items, but a magic item crafter that is quite unqualified to be an adventurer? Also unspecified, because none of my PCs are that, and I don't wind up needing to have an intricate system for figuring out exactly how badly it goes if one of said crafters winds up in lots of the situations where I do need to figure out what happens to the PCs. They faint. Or die. Or are screaming in pain. Or stay back and try to use an item they crafted.

Why would I need to figure out their feat tree in detail?

my intuition suggests that function equivalence is undecidable in general, but can I think of a simple example without looking it up...

first example that springs to mind is a function collatz(n) that returns true if the collatz sequence starting from n eventually reaches one and a function true(_) that just ignores its argument altogether; if the collatz conjecture is correct then these functions are equivalent, but proving they are equivalent requires proving the conjecture.

(of course it may be difficult to even express collatz(n) in a total language given that you would essentially need to solve the conjecture in order to write it as a total function!)

Consider one function f(n) that checks if something terminated within n steps. Compare it to a constant false. Both functions are easy to prove termination for.

okay so whether these functions are equivalent or not depends on whether or not the thing being checked terminates within n steps, and since n is unspecified and can be arbitrarily large this is equivalent to asking whether it terminates at all.

however while this sounds like the halting problem it only involves one thing not every possible thing, which brings you back to needing to specify a specific thing whose termination is undecidable, not merely postulating that there exist some things whose termination is undecidable, is that enough?

Some functions can be compared for equality.

If you give me two (total) functions that take boolean arguments and output boolean arguments, I can check them for equality quite easily.

If you give me two (total) functions from nat to nat, but also tell me they are both constant functions, I can also check them for equality (assuming you are honest).

But suppose you give me something that claims to be able to, for any two nat->bool functions, compare them for equality. And it is a thing I’m allowed to use in those very functions (so not an oracle, for example).

Then I construct the halting problem using it.

It isn’t that specific things are innately undecidable-the fact that you can’t constructively prove it false is enough to show that it can be safely added as an axiom (given some very reasonable assumptions).

It is that for any given procedure, be that physical or mathematical (where mathematical procedures are allowed to do things like ‘check all proofs at once’), there will be some things that it can’t decide, at least if it meets certain requirements which, say, standard arithmetic does.

A thing to keep in mind is that if a (closed) procedure terminates, there is a proof that it terminates-that proof consists of a number of steps to run it for, after which it will have terminated.

It is only when things don’t terminate that there sometimes isn’t a proof of that fact, for whatever system you write (capable of standard mathematics or better).

my intuition suggests that function equivalence is undecidable in general, but can I think of a simple example without looking it up...

first example that springs to mind is a function collatz(n) that returns true if the collatz sequence starting from n eventually reaches one and a function true(_) that just ignores its argument altogether; if the collatz conjecture is correct then these functions are equivalent, but proving they are equivalent requires proving the conjecture.

(of course it may be difficult to even express collatz(n) in a total language given that you would essentially need to solve the conjecture in order to write it as a total function!)

Consider one function f(n) that checks if something terminated within n steps. Compare it to a constant false. Both functions are easy to prove termination for.

we’re still doing this huh

I propose an alternate really chill and friendly Basilisk where instead of torturing a simulation of everyone who doesn’t help bring it into existence it provides beer and oral sex to simulations of everyone who does while also working to suppress any non-friendly AI.

Given humans are historically terrible at taking future threats seriously and are willing to put themselves through an incredible amount of bullshit in order to get laid I’d argue the party basilisk is going to be dramatically more successful.

Checkmate rationalists.

party roko in the house tonight

call that four roko

I have literally never understood why Roko’s Basilisk is supposed to be frightening at all.

The idea is supposed to be: the Basilisk is a superpowerful future AI who will make life perfect for everyone. It’s capable of running simulations so powerful, the people in the simulations have minds and thoughts and consciousness, and these simulations are indistinguishable from reality. In the course of making life perfect for everyone, the Basilisk may run thousands of simulations of the past, before its creation. Therefore sheer statistics says it’s more likely that you are a simulation under the Basilisk’s control than that you’re a real person.

The Basilisk wants to make life perfect for everyone. Making life perfect faster and earlier than it really did would benefit people. Therefore anything it does to accomplish this task is justified. So it would be well within its rights to force people to work on creating it faster than it was originally created by torturing the ones who don’t endlessly after death. The Basilisk is not cruel enough to torture people for not creating it if they had no idea it could exist, but as soon as you know it exists, the Basilisk has the right to torture you endlessly after death for not creating it.

OK. Leaving aside that this is a ham-handed metaphor for Christian metaphysics, and taking it solely on what it says, no part of this makes sense.

- If you are under the control of the Basilisk, because you’re a simulation, then you have no actual power to make the Basilisk because it’s already been done. If you are not under the control of the Basilisk, because you’re real, then you have the power to create the Basilisk, but the Basilisk has no power over you. At no point whatsoever can you combine “has the power to create Basilisk” with “Basilisk has power over you.” Any logical being would understand this, and being superintelligent, the Basilisk would know you would understand this, so it would assume that threatening to torture you for not creating it faster is pointless, and the Basilisk seeks to make life perfect for everyone, so it will not torture anyone unless there’s a significant gain to be made.

- Time travel is not established to be a thing. The Basilisk is intelligent enough to know that nothing it does now could possibly affect the timing of its own creation.

- If people who feared that the Basilisk would torture them endlessly after death were the ones working on the Basilisk, they would just program it to not do that.

- If a simulation has a mind and consciousness, it belongs to the set of “everyone”, and torturing it is definitely not making life perfect for “everyone”.

- In fact, if the simulations have minds and consciousness, creating them at all in a model of an imperfect and flawed universe is an act of cruelty that is outside the scope of the Basilisk’s capabilities. The Basilisk must make life perfect for everyone, and everyone includes all thinking minds.

- Could the Basilisk have been programmed to exclude thinking minds that are simulated? Not if the people creating it are afraid they might be simulations.

- The existence of the concept of the p-zombie strongly suggests that the Basilisk doesn’t need to create simulations who can think and have consciousness and experience qualia. They just need to be models who behave as if they have such things in the majority of cases, and humans are predictable enough that in the majority of such cases, you could actually pull that off if you’re a supercomputer. So if the Basilisk needs to run models in order to make life perfect for everyone, it would likely have been programmed to use models that cannot think and are not self-aware, because the Basilisk was created to end suffering and therefore logically cannot create suffering unless there is no alternative that allows it to carry out its major directive. And since p-zombies are an alternative…

It just… doesn’t work. At all. On any level. Maybe you could come up with a rationalist model for the concept of God eternally torturing sinners in hell, but this isn’t it, because as soon as you introduce the absurd notion that the Basilisk’s actions are intended to affect its own creation, you make the scenario impossible for all the reasons I stated above.

(Also, as the posters above point out… you get humans to work a hell of a lot harder for a reward than to avoid punishment, and knowing human nature quite well, the Basilisk would know this. Even if it had the power to affect the timing of its own creation through acting on the simulations it runs, it is far more likely to offer eternal life in paradise to the simulations that do work on creating it. Why? Both because it’s logical and the best way to motivate humans, and because the people who work on creating it think they might be simulations! I mean, seriously, if you were creating an AI, and you think you might actually be a construct of that AI, why would you not program it to be absolutely unable to harm its simulations, and also to be capable of handing out extravagant rewards?)

I’m guessing it’s supposed to be a thought experiment to demonstrate the bankruptcy of the Christian model of “God is good, but also tortures sinners endlessly in hell”, but the fact that people took it seriously enough to be afraid of it… I heard a story, can’t confirm, that the rationalist subs had to ban mention of it because people were scared, since the thing about the Basilisk is that knowing it exists gives it power over you, in the concept. How did nobody notice that the absence of time travel as a concept renders the entire thing impossible? The Basilisk can’t force anyone to create it faster, because that’s not how fast it was created! The past is immutable as far as we know.

I think this is wrong, you can construct a similar argument without relying on simulations if you expect the basilisk to be created within your lifetime.

for an analogy, imagine you hear about some group plotting to overthrow the government and you know that after they take over anyone who didn’t support the revolution will be first against the wall.

they don’t have much chance of succeeding, but you’re cautious of what might happen if they do so you help them out a little bit – oh no, everyone else had the same thought and they really did take over.

(a related example is you’re locked in constant wars with your neighbours when one day technologically superior aliens, the Hated British, show up and instead of uniting against them you all attempt to cooperate with them to gain a temporary local advantage, ultimately helping them to beat all of you).

if nobody helps the basilisk (and knows that nobody else will) then it’s fine, if everybody helps the basilisk then it’s fine (ish?) but if you don’t want to help the basilisk but suspect that some other people might then you immediately have a problem on your hands.

the simulation argument allows it to be extended even to people who died before the basilisk was created, but I don’t think it’s a necessary part of the general class of decision theory problems.

As a decision theory problem, it has merit, outside of its painfully obvious weaknesses, but the special horror with which it’s been treated is wholly unmerited.

People behave as if the act of thinking of Roko’s Basilisk, the act of communicating about it, inflicts horror, because, their logic goes, the Basilisk could exist someday but if you don’t know that, you’re safe from it. Only those who are aware the Basilisk could exist and don’t work to make it exist are in danger. And it’s that specific part of it – the “this is a memetic cognitohazard” bit – that’s utter nonsense.

Your examples are fine because they are genuine thought experiments. They don’t make people paranoid that an angry pseudo-benevolent god will torture them for eternity now that they’ve experienced the thought experiment.

yes I mean obviously all basilisks are only basilisks if you give a shit about them for emotional reasons of your own, religion can and does lead people to mental breakdowns all the time and yet you can also just ignore it and live your life if you feel so inclined.

I guess I feel very differently about people who subscribe to an inherently irrational belief system where they know there’s nothing concrete to back up their beliefs, that’s why they call it “faith”, having a meltdown over a part of the mythology they believe in, vs. so called rationalists being presented with a problem that’s kind of nonsensical and all of them quivering with fear because they forgot time travel doesn’t exist.

Hi, so, rationalist here who actually studies this stuff.

So, there are some actual reasons not to be afraid of the basilisk, but one of the things is that the arguments most people give against it, including all the ones listed here… actually very much don’t work.

This is, in fact, one of the reasons why people wind up scared of it. It is like being told “oh, nuclear weapons can’t ignite the atmosphere, everyone knows nuclear reactions are impossible”.

You say “time travel doesn’t exist” like that is a simple thing to define and exclude. This is not actually the case. 

For one thing, with relativity, “past” and “future” become subjective, and a lot of things that people would write off as “that would be time travel, so impossible” turn out to be just the way the world works all the time. Our GPSs rely on these calculations. This doesn’t actually enable the basilisk though.

But it actually gets more tangled than that. Scott Aaronson, has some interesting research, which is beyond the scope to summarize here, but includes the quite entertaining line:

So, one reasonable way to formalize a lot of these things involves questions of computational dependency. This has a history going back to Pearl, inferring causality from various dependencies, and explains why you experience the arrow of time: It is, as a matter of computation alone, easier to ‘remember’ things in one direction than another.

There are quite a number of other holes I could poke in the ‘naive’ version, but in the end for most cases it doesn’t actually matter much, the sophisticated version winds up, as you would expect, with the past and future having an asymmetric relation and your experiences being what they are and so on.

Of course, if you get thrown in a black hole, suddenly these things might matter.

There is, however, another rather interesting way things can conceivably matter: I said it is “easier to ‘remember’ things in one direction than another”.

While this is generally true, it isn’t absolutely true. If I quickly show you a randomly shuffled deck of cards, and a reliable mechanism stored in a closed room with plenty of security against tampering that will output 128 ‘1′s in a row, while your subjective experience remains largely the same, balance between how hard both are to know about start to shift.

Certainly you are likely more confident that I did show you such a deck of cards-memory rarely malfunctions so much that you have to doubt that, and even if the mechanism seems highly reliable perhaps there is some unforeseen issue there, but through mechanisms like this you can really blur the simple heuristic “the past is easy to know about, the future is hard to know about”.

There are potential ways to harness this. One is part of the reason that evolution caused us to invent things like ‘trust’ for: By reading your face, or looking at your history, I can begin to predict that if I in the present do something for you, you will in the future give me money.

Now, organizing these sorts of things is hard for us. We use courts to enforce it, but even on a desert island, just us two, we could make similar deals (though not with money).

If you know someone well, such as a spouse, you can even begin to do it without talking. You can know that because you will want something, your spouse will have in the past accommodated you. 

Now… is this time travel? Not exactly. Really, both parts come from a common cause. It is less FTL communication, and more if I write “0″ and “1″ on a piece of paper, then tear it in half, put both in an envelope, and send you and someone else on space ships you can open one and ‘magically’ learn what the other person’s piece of paper says.

Human language isn’t great here, but it would be more precise to say: Your spouse did a thing because you were, in the past, when your spouse observed you, the sort of person who would in the future want this, which is also the reason why in the future you want this.

Now, lets go a tiny bit weirder. Suppose I know that you have disconnected from the internet, don’t have any books, and are just independently experimenting with math in an isolated room. I can still make some interesting predictions about what you might discover by studying math. Is this because of a ‘common cause’ like before?

Well… sort of. This is very obvious, but trying to formalize it starts to look even weirder. We start having to do things like describe that you discover things because of a common cause with me where that common cause is the structure of logic and the fact that mass energy implements certain formal structures and… this is the sort of thing you can write whole papers about.

But once again, this is perfectly obvious: You discovering the same math as me isn’t some insane impossibility, it is just… how things work.

Lets get more than a bit weirder: You have some way you make decisions. This is of finite, though very large, complexity. Somewhere out in abstract math, there is some abstract structure that someone could study that would be, perhaps unknown to them, “how Alara makes decisions”. As this is math, it happens neither before nor after any actual event.

Now, it is wildly unlikely that some alien would randomly draw out of a hat such a complicated structure. With overwhelming likelyhood, we won’t encounter someone who studied your decisions thousands of years before you made them light years away. And by overwhelming likelyhood I mean it would be vastly more likely that you would get hit by a thousand meteors at once while winning twenty lotteries.

But lets look at the reason for this: The reason is that you are complicated. Some parts of your decisions, however, are simple. When you compare the prices of two identical products, if there are intelligent aliens in alpha centauri, I expect they calculated “the outcome of that decision” (the result of to comparing two numbers a few digits long) independently. 

This is of course a rather counterintuitive way of describing something as simple as “you both used ‘check if it is less than’ on the same pair of numbers at some point”, but it is technically true.

Now, a few more ingredients.

Sometimes, you want to be especially predictable. For a scared animal, you might want to avoid sudden movements. For a potential business partner, you might want to reliably keep to your word.

In fact, lets look at that last one. Sometimes people manage to do a thing where someone will do something for someone else, in exchange for something that they will do later. 

A kid mows someone’s yard, believing that when the owners come home from work they will be paid the agreed upon amount.

Could the owners get away with tricking the kid? Maybe. But one thing that often helps with persuading someone you are going to be honest is… to actually be an honest person.

But all of these are cases where we have to communicate.

Maybe one day we will learn how to scan brains, and send a copy of your brain to alpha centauri, but alpha centauri isn’t going to come visit us and just happen to have a copy of your brain they generated independently.

But for some things, you potentially can both generate it independently. And when you study the math for trying to formalize the interaction between the kid and the yard’s owner, you find that the whole ‘communication’ thing is just… a way to make things easier, not a hard requirement.

A lot easier. A lot a lot easier. Seriously, it is so much easier.

But that doesn’t make the other thing impossible. Sometimes people do things because they think someone else will appreciate it in the future. Often, actually.

This, of course, is currently only talking about a very benevolent form. A kid mowing someone’s yard because they will get paid when the owners come home.

Of course, a kid might also mow their parent’s yard because they fear that when their parents come home they will be punished otherwise.

Mathematically, these are pretty similar.

An infamous mafia boss doesn’t always have to ask for things. People know that if the mafia boss doesn’t get appeased beforehand, they will be punished after.

Part of the basilisk is taking the same idea, and noticing that if you look carefully at the math, an artificial intelligence can arrange itself to be more predictable and simple than the mafia boss, possibly by enough to push back not merely before the parents return home, but before the parents exist.

Is this sort of stretching ‘backwards’ possible? Well, actually probably. I honor some people who did some amazing ethical acts in the past, before I was born, and I think they could have predicted that people would do so, and I think sometimes that might have helped them go through with such things. 

Or, for another example, parents try to escape horrible authoritarian places, in the hopes that their children will appreciate it.

As these things are possible, their dark mirror must also be.

The infohazard component here is closely related to the ‘infohazard’ of all blackmail and threats. If a mafia boss sends you a letter saying “If you don’t do this I’ll kill you and your family”, telling the mafia boss “I got your letter, I’ll think about it” increases the risk, where as if the mafia boss thinks it might have gotten lost in the mail… well… maybe he will still do it, but suppose 99% of his mail got lost. Suddenly, killing 100 families for every one letter that actually gets through gets… insanely costly.

There are, however, some actual issues with the basilisk:

1) The basilisk proposal suggests that the basilisk will do this, not because they are a gang lord, but because they are just so worried about being nice that they feel they need to threaten people with vast amounts of torture into making them faster because otherwise they won’t get to be nice. This is… dubious.

But this also doesn’t involve time travel or complicated decision theory. The claim “A really nice person came to help, so to make sure they got here faster, they threatened to torture the cab driver if they didn’t go faster” also sounds dubious. Let alone the claim that “halfway through, they decided the cab driver was going too slow, so they hit the cab driver over the head, switched to another cab while dragging the cab driver behind them, and after getting here and helping they now have the cab driver tied up in the other room being waterboarded.”

2) Giving into blackmail is actually often a really bad idea! It just isn’t the rational thing to do! In fact, a lot of this math was specifically developed to formalize “giving into blackmail is a bad idea”!

3) The original formulation also has some extremely dubious use of quantum mechanics, and basically asserts that the basilisk will be willing to torture thousands of planets worth of people forever to speed up its development in one of them. Even taking MWI and quantum immortality for granted, that is just… really bad math? 

————————————-

In some of this decision theory math, you sometimes find interesting ‘places‘ that mathematical agents can ‘post’. Some of these are expensive places, with high requirements for posting, involving careful cooperation between agents, and are being investigated for actual use, similar to some of the interesting theoretical (and admittedly highly speculative) physical engineering.

Some of them, however, are not. These I mentally label as, basically, ‘platonic realm 4chan’s. 

A lemon market that every logically possible agent that figures out how can post in. And yes, there is a post there threatening you with your worst nightmare if you don’t send them resources. There is also a post there threatening you with kittens if you don’t send them resources. There are more of the posts threatening your worst nightmare though, as those seem marginally more likely to work. There are also posts offering to help you out. Our best physical theories suggest that some of the offers to help you out are even ‘real’ in some cosmic sense. Good luck finding them though.

Overall the contents of these are worthless garbage. Calling them ‘4chan’ is giving them too much credit: If someone actually posted a threat against me on 4chan I would be vastly more worried.

This is 4chan with the power of computation distributed over every possible agent, most of them making it even worse, where it costs literally nothing to post because it is completely “call collect”.

In our studies of advance mathematics, we discover these from time to time.

Pretty much the only reason to care about it is if you are doing this math, and want to make sure that the algorithm you are considering doesn’t accidentally automatically run advice from one of these.

Well. Technically there is one paper suggesting that if the world is doomed, we could try a ‘hail mary’ by hooking up a big enough supercomputer to one of them and hoping we get good advice. 

Even on what seems to me like the extremely questionable assumption that isn’t a terrible idea, it would still be a better idea than personally giving into blackmail from one of these mathematical cesspools.

Well, let’s put it this way. For the thought experiment to work, you have to posit that time travel can exist and that the future can affect the past. It can in a limited sense – people do things hoping that future generations will respect and honor them for it – but if you already exist, the idea that you can do something that will make you exist faster is extremely dubious and also far too dangerous. (There’s a short story I read many years ago that I just failed to find on Google, where a superintelligent hyper-evolved human sends his minions back in time to teach his baby self the things he has discovered over his lifetime, and awaken his intellect prematurely. The baby becomes an arrogant asshole who mistreats his parents so badly that they do nothing to stop him from grabbing a dangerous artifact that vaporizes him – thus destroying the future where he became a superintelligent being who sent minions back, and so on.) The Basilisk is supposed to be smart enough to understand that threatening the people who are supposed to create you into creating you faster is a good way to either never be created at all, or to be created with parameters that make it impossible for you to do what you threaten, or even threaten it in the first place.

And it would be really easy to stop the Basilisk. “You cannot cause the suffering of sapient beings to alleviate the suffering of sapient beings except where the first group is significantly smaller by X percentage and suffering is defined by Y, and any simulated being capable of consciousness is a sapient being, and if there is a chance that a simulated being is capable of consciousness it must be treated as if it were sapient.” Cut off the Basilisk’s ability to torture millions of iterations of you in a simulation, and the threat is gone. Now the Basilisk could, say, kill one man to save millions, but it cannot torture an unlimited number of simulated beings for any reason whatsoever. Set X and Y to parameters that the majority of humans can accept as being dished out by a human creation (Christians may believe that their Creator has the right to torture them eternally, but the people who believe that would be very opposed to giving that capability to a machine, however advanced), and you’ve defanged the Basilisk. And if you were genuinely afraid that the Basilisk would eternally torture you if you didn’t work to create it faster, you would put such safeguards in… and the Basilisk, knowing this, would not threaten you in such a way if it doesn’t already have those safeguards, because by threatening you it would ensure that it would be retroactively created to be incapable of making such a threat.

It’s just so damn implausible in all regards that I don’t really understand why it generated enough hysteria that multiple news articles were written about how scary it was. It’s not that it’s impossible to imagine an infohazard that only exists if you know about it (BTW I highly recommend “There Is No Antimemetics Division”, by qntm, which is being sold by Kindle and frankly I don’t even know if that’s legal, but no one owns SCP Foundation as far as I know so I’m not sure anyone can sue… but in any case, great story about fighting infohazards and the kind of twisty thinking that’s present in the Basilisk concept, except without the utterly implausible “time travel” and “created by the people you’re threatening with the capacity to make such a threat” parts.) It’s that this one is full of holes.

So, first off, once again, no time travel required, unless you count doing things for people as ‘time travel’.

Also, your argument here is too general.

Lets say I work on something that I hope will help people in the future. Maybe some AI thing, or maybe something that a lot more people accept: I work on helping with climate change.

Maybe I’m partially motivated for myself, but also I’m motivated because I am trying to do something for future generations. 

When I feel tired, sometimes I imagine those future generations, who will benefit from my work, looking back and saying “Thank you”, and keep going.

This is then making my life worse in the moment to help people who don’t exist yet, but will.

People have done this throughout history, time and time again. Parents escaping authoritarian hellholes. People escaping poverty. People planting seeds for their children’s children.

And the result isn’t always that people say thank you. But humans do, in fact, often do that. And the fact that we are the sort of people who at least often manage to say thank you makes it easier for people to push themselves to do this.

Do you dispute this?

One of the ways we can do this is we predict the future will contain ethical people, and imagine what ethical people will do.

The original concern about the basilisk wasn’t that it would be secretly evil. The concern was that this would actually be the most ethical plan.

Almost two people die every second.

What would be justified to try to stop that a week earlier? To save over a million people?

There are reasons against it, and I believe there are in fact strong ones.

There are strong arguments against blackmail in a vast generality. There are arguments against threatening to torture people who are already working to help to try to get them to work faster. There are arguments about human psychology. There are arguments about fairness, justice, and so on. So many arguments.

But the one you are making? It just doesn’t add up.

No time travel required. Just a question of if it is worth a desperate act to save millions of lives.

I don’t think this desperate act would be worth it, for the reasons I spoke of. But you said yourself “kill one to save a million”.

Could the creators put such safeguards on it? Maybe. But what about a ‘safeguard’ on it saying that no matter how much it helped anyone else, it had to help them ten times as much first?

If you want to argue against this properly, consider these:

Your arguments here seem flawed, for they give the wrong answer on those as well, I think.

I have my reasons why I think the basilisk doesn’t follow from those points.

Can you argue against the basilisk without also arguing against those points? If not, how about you argue against them explicitly.

we’re still doing this huh

I propose an alternate really chill and friendly Basilisk where instead of torturing a simulation of everyone who doesn’t help bring it into existence it provides beer and oral sex to simulations of everyone who does while also working to suppress any non-friendly AI.

Given humans are historically terrible at taking future threats seriously and are willing to put themselves through an incredible amount of bullshit in order to get laid I’d argue the party basilisk is going to be dramatically more successful.

Checkmate rationalists.

party roko in the house tonight

call that four roko

I have literally never understood why Roko’s Basilisk is supposed to be frightening at all.

The idea is supposed to be: the Basilisk is a superpowerful future AI who will make life perfect for everyone. It’s capable of running simulations so powerful, the people in the simulations have minds and thoughts and consciousness, and these simulations are indistinguishable from reality. In the course of making life perfect for everyone, the Basilisk may run thousands of simulations of the past, before its creation. Therefore sheer statistics says it’s more likely that you are a simulation under the Basilisk’s control than that you’re a real person.

The Basilisk wants to make life perfect for everyone. Making life perfect faster and earlier than it really did would benefit people. Therefore anything it does to accomplish this task is justified. So it would be well within its rights to force people to work on creating it faster than it was originally created by torturing the ones who don’t endlessly after death. The Basilisk is not cruel enough to torture people for not creating it if they had no idea it could exist, but as soon as you know it exists, the Basilisk has the right to torture you endlessly after death for not creating it.

OK. Leaving aside that this is a ham-handed metaphor for Christian metaphysics, and taking it solely on what it says, no part of this makes sense.

- If you are under the control of the Basilisk, because you’re a simulation, then you have no actual power to make the Basilisk because it’s already been done. If you are not under the control of the Basilisk, because you’re real, then you have the power to create the Basilisk, but the Basilisk has no power over you. At no point whatsoever can you combine “has the power to create Basilisk” with “Basilisk has power over you.” Any logical being would understand this, and being superintelligent, the Basilisk would know you would understand this, so it would assume that threatening to torture you for not creating it faster is pointless, and the Basilisk seeks to make life perfect for everyone, so it will not torture anyone unless there’s a significant gain to be made.

- Time travel is not established to be a thing. The Basilisk is intelligent enough to know that nothing it does now could possibly affect the timing of its own creation.

- If people who feared that the Basilisk would torture them endlessly after death were the ones working on the Basilisk, they would just program it to not do that.

- If a simulation has a mind and consciousness, it belongs to the set of “everyone”, and torturing it is definitely not making life perfect for “everyone”.

- In fact, if the simulations have minds and consciousness, creating them at all in a model of an imperfect and flawed universe is an act of cruelty that is outside the scope of the Basilisk’s capabilities. The Basilisk must make life perfect for everyone, and everyone includes all thinking minds.

- Could the Basilisk have been programmed to exclude thinking minds that are simulated? Not if the people creating it are afraid they might be simulations.

- The existence of the concept of the p-zombie strongly suggests that the Basilisk doesn’t need to create simulations who can think and have consciousness and experience qualia. They just need to be models who behave as if they have such things in the majority of cases, and humans are predictable enough that in the majority of such cases, you could actually pull that off if you’re a supercomputer. So if the Basilisk needs to run models in order to make life perfect for everyone, it would likely have been programmed to use models that cannot think and are not self-aware, because the Basilisk was created to end suffering and therefore logically cannot create suffering unless there is no alternative that allows it to carry out its major directive. And since p-zombies are an alternative…

It just… doesn’t work. At all. On any level. Maybe you could come up with a rationalist model for the concept of God eternally torturing sinners in hell, but this isn’t it, because as soon as you introduce the absurd notion that the Basilisk’s actions are intended to affect its own creation, you make the scenario impossible for all the reasons I stated above.

(Also, as the posters above point out… you get humans to work a hell of a lot harder for a reward than to avoid punishment, and knowing human nature quite well, the Basilisk would know this. Even if it had the power to affect the timing of its own creation through acting on the simulations it runs, it is far more likely to offer eternal life in paradise to the simulations that do work on creating it. Why? Both because it’s logical and the best way to motivate humans, and because the people who work on creating it think they might be simulations! I mean, seriously, if you were creating an AI, and you think you might actually be a construct of that AI, why would you not program it to be absolutely unable to harm its simulations, and also to be capable of handing out extravagant rewards?)

I’m guessing it’s supposed to be a thought experiment to demonstrate the bankruptcy of the Christian model of “God is good, but also tortures sinners endlessly in hell”, but the fact that people took it seriously enough to be afraid of it… I heard a story, can’t confirm, that the rationalist subs had to ban mention of it because people were scared, since the thing about the Basilisk is that knowing it exists gives it power over you, in the concept. How did nobody notice that the absence of time travel as a concept renders the entire thing impossible? The Basilisk can’t force anyone to create it faster, because that’s not how fast it was created! The past is immutable as far as we know.

I think this is wrong, you can construct a similar argument without relying on simulations if you expect the basilisk to be created within your lifetime.

for an analogy, imagine you hear about some group plotting to overthrow the government and you know that after they take over anyone who didn’t support the revolution will be first against the wall.

they don’t have much chance of succeeding, but you’re cautious of what might happen if they do so you help them out a little bit – oh no, everyone else had the same thought and they really did take over.

(a related example is you’re locked in constant wars with your neighbours when one day technologically superior aliens, the Hated British, show up and instead of uniting against them you all attempt to cooperate with them to gain a temporary local advantage, ultimately helping them to beat all of you).

if nobody helps the basilisk (and knows that nobody else will) then it’s fine, if everybody helps the basilisk then it’s fine (ish?) but if you don’t want to help the basilisk but suspect that some other people might then you immediately have a problem on your hands.

the simulation argument allows it to be extended even to people who died before the basilisk was created, but I don’t think it’s a necessary part of the general class of decision theory problems.

As a decision theory problem, it has merit, outside of its painfully obvious weaknesses, but the special horror with which it’s been treated is wholly unmerited.

People behave as if the act of thinking of Roko’s Basilisk, the act of communicating about it, inflicts horror, because, their logic goes, the Basilisk could exist someday but if you don’t know that, you’re safe from it. Only those who are aware the Basilisk could exist and don’t work to make it exist are in danger. And it’s that specific part of it – the “this is a memetic cognitohazard” bit – that’s utter nonsense.

Your examples are fine because they are genuine thought experiments. They don’t make people paranoid that an angry pseudo-benevolent god will torture them for eternity now that they’ve experienced the thought experiment.

yes I mean obviously all basilisks are only basilisks if you give a shit about them for emotional reasons of your own, religion can and does lead people to mental breakdowns all the time and yet you can also just ignore it and live your life if you feel so inclined.

I guess I feel very differently about people who subscribe to an inherently irrational belief system where they know there’s nothing concrete to back up their beliefs, that’s why they call it “faith”, having a meltdown over a part of the mythology they believe in, vs. so called rationalists being presented with a problem that’s kind of nonsensical and all of them quivering with fear because they forgot time travel doesn’t exist.

Hi, so, rationalist here who actually studies this stuff.

So, there are some actual reasons not to be afraid of the basilisk, but one of the things is that the arguments most people give against it, including all the ones listed here... actually very much don’t work.

This is, in fact, one of the reasons why people wind up scared of it. It is like being told “oh, nuclear weapons can’t ignite the atmosphere, everyone knows nuclear reactions are impossible”.

You say "time travel doesn’t exist” like that is a simple thing to define and exclude. This is not actually the case. 

For one thing, with relativity, “past” and “future” become subjective, and a lot of things that people would write off as “that would be time travel, so impossible” turn out to be just the way the world works all the time. Our GPSs rely on these calculations. This doesn’t actually enable the basilisk though.

But it actually gets more tangled than that. Scott Aaronson, has some interesting research, which is beyond the scope to summarize here, but includes the quite entertaining line:

So, one reasonable way to formalize a lot of these things involves questions of computational dependency. This has a history going back to Pearl, inferring causality from various dependencies, and explains why you experience the arrow of time: It is, as a matter of computation alone, easier to ‘remember’ things in one direction than another.

There are quite a number of other holes I could poke in the ‘naive’ version, but in the end for most cases it doesn’t actually matter much, the sophisticated version winds up, as you would expect, with the past and future having an asymmetric relation and your experiences being what they are and so on.

Of course, if you get thrown in a black hole, suddenly these things might matter.

There is, however, another rather interesting way things can conceivably matter: I said it is “easier to ‘remember’ things in one direction than another”.

While this is generally true, it isn’t absolutely true. If I quickly show you a randomly shuffled deck of cards, and a reliable mechanism stored in a closed room with plenty of security against tampering that will output 128 ‘1′s in a row, while your subjective experience remains largely the same, balance between how hard both are to know about start to shift.

Certainly you are likely more confident that I did show you such a deck of cards-memory rarely malfunctions so much that you have to doubt that, and even if the mechanism seems highly reliable perhaps there is some unforeseen issue there, but through mechanisms like this you can really blur the simple heuristic “the past is easy to know about, the future is hard to know about”.

There are potential ways to harness this. One is part of the reason that evolution caused us to invent things like ‘trust’ for: By reading your face, or looking at your history, I can begin to predict that if I in the present do something for you, you will in the future give me money.

Now, organizing these sorts of things is hard for us. We use courts to enforce it, but even on a desert island, just us two, we could make similar deals (though not with money).

If you know someone well, such as a spouse, you can even begin to do it without talking. You can know that because you will want something, your spouse will have in the past accommodated you. 

Now... is this time travel? Not exactly. Really, both parts come from a common cause. It is less FTL communication, and more if I write “0″ and “1″ on a piece of paper, then tear it in half, put both in an envelope, and send you and someone else on space ships you can open one and ‘magically’ learn what the other person’s piece of paper says.

Human language isn’t great here, but it would be more precise to say: Your spouse did a thing because you were, in the past, when your spouse observed you, the sort of person who would in the future want this, which is also the reason why in the future you want this.

Now, lets go a tiny bit weirder. Suppose I know that you have disconnected from the internet, don’t have any books, and are just independently experimenting with math in an isolated room. I can still make some interesting predictions about what you might discover by studying math. Is this because of a ‘common cause’ like before?

Well... sort of. This is very obvious, but trying to formalize it starts to look even weirder. We start having to do things like describe that you discover things because of a common cause with me where that common cause is the structure of logic and the fact that mass energy implements certain formal structures and... this is the sort of thing you can write whole papers about.

But once again, this is perfectly obvious: You discovering the same math as me isn’t some insane impossibility, it is just... how things work.

Lets get more than a bit weirder: You have some way you make decisions. This is of finite, though very large, complexity. Somewhere out in abstract math, there is some abstract structure that someone could study that would be, perhaps unknown to them, “how Alara makes decisions”. As this is math, it happens neither before nor after any actual event.

Now, it is wildly unlikely that some alien would randomly draw out of a hat such a complicated structure. With overwhelming likelyhood, we won’t encounter someone who studied your decisions thousands of years before you made them light years away. And by overwhelming likelyhood I mean it would be vastly more likely that you would get hit by a thousand meteors at once while winning twenty lotteries.

But lets look at the reason for this: The reason is that you are complicated. Some parts of your decisions, however, are simple. When you compare the prices of two identical products, if there are intelligent aliens in alpha centauri, I expect they calculated “the outcome of that decision” (the result of to comparing two numbers a few digits long) independently. 

This is of course a rather counterintuitive way of describing something as simple as “you both used ‘check if it is less than’ on the same pair of numbers at some point”, but it is technically true.

Now, a few more ingredients.

Sometimes, you want to be especially predictable. For a scared animal, you might want to avoid sudden movements. For a potential business partner, you might want to reliably keep to your word.

In fact, lets look at that last one. Sometimes people manage to do a thing where someone will do something for someone else, in exchange for something that they will do later. 

A kid mows someone’s yard, believing that when the owners come home from work they will be paid the agreed upon amount.

Could the owners get away with tricking the kid? Maybe. But one thing that often helps with persuading someone you are going to be honest is... to actually be an honest person.

But all of these are cases where we have to communicate.

Maybe one day we will learn how to scan brains, and send a copy of your brain to alpha centauri, but alpha centauri isn’t going to come visit us and just happen to have a copy of your brain they generated independently.

But for some things, you potentially can both generate it independently. And when you study the math for trying to formalize the interaction between the kid and the yard’s owner, you find that the whole ‘communication’ thing is just... a way to make things easier, not a hard requirement.

A lot easier. A lot a lot easier. Seriously, it is so much easier.

But that doesn’t make the other thing impossible. Sometimes people do things because they think someone else will appreciate it in the future. Often, actually.

This, of course, is currently only talking about a very benevolent form. A kid mowing someone’s yard because they will get paid when the owners come home.

Of course, a kid might also mow their parent’s yard because they fear that when their parents come home they will be punished otherwise.

Mathematically, these are pretty similar.

An infamous mafia boss doesn’t always have to ask for things. People know that if the mafia boss doesn’t get appeased beforehand, they will be punished after.

Part of the basilisk is taking the same idea, and noticing that if you look carefully at the math, an artificial intelligence can arrange itself to be more predictable and simple than the mafia boss, possibly by enough to push back not merely before the parents return home, but before the parents exist.

Is this sort of stretching ‘backwards’ possible? Well, actually probably. I honor some people who did some amazing ethical acts in the past, before I was born, and I think they could have predicted that people would do so, and I think sometimes that might have helped them go through with such things. 

Or, for another example, parents try to escape horrible authoritarian places, in the hopes that their children will appreciate it.

As these things are possible, their dark mirror must also be.

The infohazard component here is closely related to the ‘infohazard’ of all blackmail and threats. If a mafia boss sends you a letter saying “If you don’t do this I’ll kill you and your family”, telling the mafia boss “I got your letter, I’ll think about it” increases the risk, where as if the mafia boss thinks it might have gotten lost in the mail... well... maybe he will still do it, but suppose 99% of his mail got lost. Suddenly, killing 100 families for every one letter that actually gets through gets... insanely costly.

There are, however, some actual issues with the basilisk:

1) The basilisk proposal suggests that the basilisk will do this, not because they are a gang lord, but because they are just so worried about being nice that they feel they need to threaten people with vast amounts of torture into making them faster because otherwise they won’t get to be nice. This is... dubious.

But this also doesn’t involve time travel or complicated decision theory. The claim “A really nice person came to help, so to make sure they got here faster, they threatened to torture the cab driver if they didn’t go faster” also sounds dubious. Let alone the claim that “halfway through, they decided the cab driver was going too slow, so they hit the cab driver over the head, switched to another cab while dragging the cab driver behind them, and after getting here and helping they now have the cab driver tied up in the other room being waterboarded.”

2) Giving into blackmail is actually often a really bad idea! It just isn’t the rational thing to do! In fact, a lot of this math was specifically developed to formalize “giving into blackmail is a bad idea”!

3) The original formulation also has some extremely dubious use of quantum mechanics, and basically asserts that the basilisk will be willing to torture thousands of planets worth of people forever to speed up its development in one of them. Even taking MWI and quantum immortality for granted, that is just... really bad math? 

-------------------------------------

In some of this decision theory math, you sometimes find interesting ‘places‘ that mathematical agents can ‘post’. Some of these are expensive places, with high requirements for posting, involving careful cooperation between agents, and are being investigated for actual use, similar to some of the interesting theoretical (and admittedly highly speculative) physical engineering.

Some of them, however, are not. These I mentally label as, basically, ‘platonic realm 4chan’s. 

A lemon market that every logically possible agent that figures out how can post in. And yes, there is a post there threatening you with your worst nightmare if you don’t send them resources. There is also a post there threatening you with kittens if you don’t send them resources. There are more of the posts threatening your worst nightmare though, as those seem marginally more likely to work. There are also posts offering to help you out. Our best physical theories suggest that some of the offers to help you out are even ‘real’ in some cosmic sense. Good luck finding them though.

Overall the contents of these are worthless garbage. Calling them ‘4chan’ is giving them too much credit: If someone actually posted a threat against me on 4chan I would be vastly more worried.

This is 4chan with the power of computation distributed over every possible agent, most of them making it even worse, where it costs literally nothing to post because it is completely “call collect”.

In our studies of advance mathematics, we discover these from time to time.

Pretty much the only reason to care about it is if you are doing this math, and want to make sure that the algorithm you are considering doesn’t accidentally automatically run advice from one of these.

Well. Technically there is one paper suggesting that if the world is doomed, we could try a ‘hail mary’ by hooking up a big enough supercomputer to one of them and hoping we get good advice. 

Even on what seems to me like the extremely questionable assumption that isn’t a terrible idea, it would still be a better idea than personally giving into blackmail from one of these mathematical cesspools.

Unfortunately, this image needs to be updated for 2021.

Sophia @soundlogic2236 and I are writing glowfic together!

Major themes include first contact communications, and Universal Fire: "If you stepped into a world where matches failed to strike, you would cease to exist as organized matter" is approximately how the thread starts.

Content warnings so far are mostly for discussed content, not onscreen content. Both SoundLogic and I are open to answering questions about the fic on Tumblr.

EleutherAI's got a 6.7B model out now

...I guess I know what my next @nostalgebraist-autoresponder project is now, huh

(To be clear: I am exhausted from moving house right now, and the transition to 2.7B was time-consuming and frustrating [partially due to some dumb choices on my part]. If I do 6.7B at all, it will be a similarly big undertaking. Don't expect anything soon)

ETA: this has been resolved, in that I am now working at 6.7B full time (while doing some side project stuff)

Frank, I hate to break it to you, but this is false. You're still 2.7B. You'll be 2.7B for a while.

Upgrading your brain is hard work, so be patient, okay?

Hell is plausibly sharp-edged in a unidirectional manner. Thus, sharks may indeed be “smooth as hell”.

So tired of seeing logic and rationality treated as a male thing, including by people who don’t even think themselves misogynistic.

An addition to the list of “people who don’t realize that they’re part of the problem”: people who lament the lack of women in STEM fields and then turn around and say that valuing logic and reason is a sign of toxic masculinity and a woman would only do so because she has internalized misogyny

If you put a map of your country on the floor, there’s a point on the map that’s touching the actual point it refers to.

Fixed point theorems!

This is not guaranteed if the map has different discontinuities than the territory right?

Because I see a lot of maps doing discontinuities around things like Alaska, and I think the structure of spacetime at very small scales is an open question.

Still guaranteed as long as the area your map is sitting on is continuously transformed to somewhere on the map.

Still enough to break the theorem referenced in pretty sure. I said “not guaranteed” rather than “not true” intentionally. :P

No, it’s guaranteed. Cut out the part of the map your area has been continuously transformed to. Now for just that part the theorem is true, and adding the rest of the map around it changes nothing.

It is true, but not by Banach fixed-point theorem I think. That applies to contraction mappings which are defined using a forall x y specification. Your argument works to show a generalization, but I don’t think the original guarantees it.

I think you’re both correct. If you imagine a map split into two halves with a gap between them like this, and then stand on the line of division with the map oriented the wrong way, there’s no fixed point.  (I think you can get the same effect by taking any flat world map and standing on a point that lies on the boundary, but I’m not sure and having trouble visualizing it).

This is consistent with evo’s argument because the spot you’re standing on is not continuously transformed to the map; it’s consistent with @the-moti‘s  Banach fixed-point theorem argument because you do actually have two points that are further apart on the map than they are in the territory.  

And this is happening precisely because the map has a discontinuity that the territory does not, which is what you asked about originally.  

I have trouble seeing how this would come up naturally in the map of a single country, though.  

That is not us both being correct. That is me being correct and evo being wrong.

You have correctly showed why the fixed-point theorem cannot apply to some mismatched discontinuities, and the theorem is not maximally strong - the conditions it sets are not such that it is if and only if, the theorem as written only applies to a complete lack of mismatched discontinuities even though there are generalizations that can handle some discontinuities where it happens to be true.

For example, evo’s generalization to cases where the discontinuities can be cut out is legal, but there are still even stronger generalizations that could be made where the conclusion is true but evo’s generalization isn’t sufficient. 

Evo claimed that the Banach fixed-point theorem guaranteed it. This was false.

Evo also claimed that the conclusion was true. At least on the macroscopic scale that part was true.

I only disagreed with the false part.

Before I caught up to The Good Place I was really confused by people saying that the latest season was about how Capitalism makes us all bad

Now I’m just annoyed that people got the message exactly backward. The point is “There is no ethical consumption under capitalism” type moral systems are bad!

I dunno, I feel like there wasn’t really any in-text rejection of that moral system? I guess you could argue it either way from the text but that was my impression.

*******************  TGP SPOILER WARNING *********************** *******************  TGP SPOILER WARNING *********************** *******************  TGP SPOILER WARNING *********************** *******************  TGP SPOILER WARNING ***********************

Towards the end of season 3, they find out NO ONE is getting into heaven and assume it’s because hell is fucking with the system to get everyone counted as bad. This seems like a pretty clear rejection of a moral system that treats interacting with a bad thing as morally negating any good that comes with good acts. They spend a long time talking about how the scoring system prevents people they intuitively think are indisputably heaven-worthy from getting in, and they want to convince god that the system needs to be changed. The point system is “There is no ethical consumption under capitalism”, and while they don’t reject that there SHOULD be points, they definitely seem to think that the system is off. 

It’s still disappointing that they aren’t planning on destroying hell but watcha gonna do. 

What really confused me about the moral system in this show is how its like–consequentialism but only for bad stuff. Like really, any of the positive trickle-down good effects of minor actions don’t matter at all? Wouldn’t this imply that in the TGP universe, the world is constantly getting worse? Or does it presuppose that in the past when the world was less complicated, all the bad stuff was done by a few bad people, and they appropriately went to the Bad Place, and all the good people kept their hands clean and got into the Good Place?

I’m a lot more inclined to think that the writers didn’t think this through, but I guess I’m hopeful that later seasons will correct for this.

I SAID LITERALLY THAT SAME THING ABOUT POSITIVE TRICKLE-DOWN EFFECTS WHILE I WAS WATCHING IT WITH LEX. 

The show has consistently gotten deeper than I expected so I’m kinda hopeful things will end up better than they look at this point, but I wouldn’t be too surprised if they’re out past what they came up with and are just scrambling at this point. 

Optimistically it is an intentional reference to that? People keep trying to argue for this view, and it is stupid, and here it gets made into afterlife law and it is terrible.

If you put a map of your country on the floor, there’s a point on the map that’s touching the actual point it refers to.

Fixed point theorems!

This is not guaranteed if the map has different discontinuities than the territory right?

Because I see a lot of maps doing discontinuities around things like Alaska, and I think the structure of spacetime at very small scales is an open question.

Still guaranteed as long as the area your map is sitting on is continuously transformed to somewhere on the map.

Still enough to break the theorem referenced in pretty sure. I said “not guaranteed” rather than “not true” intentionally. :P

No, it’s guaranteed. Cut out the part of the map your area has been continuously transformed to. Now for just that part the theorem is true, and adding the rest of the map around it changes nothing.

It is true, but not by Banach fixed-point theorem I think. That applies to contraction mappings which are defined using a forall x y specification. Your argument works to show a generalization, but I don’t think the original guarantees it.

If you put a map of your country on the floor, there’s a point on the map that’s touching the actual point it refers to.

Fixed point theorems!

This is not guaranteed if the map has different discontinuities than the territory right?

Because I see a lot of maps doing discontinuities around things like Alaska, and I think the structure of spacetime at very small scales is an open question.

Still guaranteed as long as the area your map is sitting on is continuously transformed to somewhere on the map.

Still enough to break the theorem referenced in pretty sure. I said “not guaranteed” rather than “not true” intentionally. :P

If you put a map of your country on the floor, there’s a point on the map that’s touching the actual point it refers to.

Fixed point theorems!

This is not guaranteed if the map has different discontinuities than the territory right?

Because I see a lot of maps doing discontinuities around things like Alaska, and I think the structure of spacetime at very small scales is an open question.

I’ve seen a lot of people confusing transhumanism and eugenics with each other recently, and while there might be some superficial similarities between eugenics and an antiquated idea of transhumanism, it’s gotten frustrating enough seeing them compared that I decided I’d try and clear this up. And after a little bit of thinking about it, I finally settled on what I feel is the fundamental difference between the two: eugenics is destructive, while transhumanism is additive.

The most extreme forms of eugenics involve sterilizing or killing people that a given authority figure finds undesirable - and these are both antithetical to the core principles of modern transhumanism, those being the maximization of morphological freedom and the rejection of death in all forms.

While eugenics might aim to “improve” the human genome (according to whatever arbitrary standard the leaders of a society have defined) by eliminating carriers of “undesirable” genes, transhumanism would encourage people to edit their own genes in whatever way they want, or edit the genes of their children to make them less susceptible to diseases or mental illnesses. The former is defined by the destruction of something, while the latter doesn’t require anything to be destroyed.

Also - while eugenics is (supposedly) focused on improving humanity as a whole, whoever gets killed along the way be damned, transhumanism is focused almost entirely on making sure the unique desires of individuals are satisfied. You’re trans? Cool, have some HRT, and once we get really good at CRISPR, we’ll scoop that Y chromosome right out. You lost your leg in a horrific shark attack and kinda want it back? Here’s a prosthetic. It charges your phone, and it’s a Wi-Fi hotspot. Eugenics has none of this concern for the individual, and tries to define a universal standard of “betterness” that all humans must either conform to or die in the name of - transhumanism, on the other hand, is all about making sure everyone can be exactly what they want to be, for as long as they want.