The fears people who like to talk about the singularity like to propose is that there will be one ‘rogue’ misaligned ASI that progressively takes over everything - i.e. all the AI in the world works against all the people.

My point is that more likely is there will be lots of ASI or AGI systems, not aligned to each other, most on the side of the humans.

I think any prediction based on a ‘singularity’ neglects to consider the physical limitations, and just how long the journey towards significant amounts of AGI would be.

The human brain has an estimated 100 trillion neuronal connections - so probably a good order of magnitude estimation for the parameter count of an AGI model.

If we consider a current GPU, e.g. the 12 GB GFX 3060, it can hold about 24 billion parameters at 4 bit quantisation (in reality a fair few less), and uses 180 W of power. So that means an AGI might use 750 kW of power to operate. A super-intelligent machine might use more. That is a farm of 2500 300W solar panels, while the sun is shining, just for the equivalent of one person.

Now to pose a real threat against the billions of humans, you’d need more than one person’s worth of intelligence. Maybe an army equivalent to 1,000 people, powered by 8,333,333 GPUs and 2,500,000 solar panels.

That is not going to materialise out of the air too quickly.

In practice, as we get closer to an AGI or ASI, there will be multiple separate deployments of similar sizes (within an order of magnitude), and they won’t be aligned to each other - some systems will be adversaries of any system executing a plan to destroy humanity, and will be aligned to protect against harm (AI technologies are already widely used for threat analysis). So you’d have a bunch of malicious systems, and a bunch of defender systems, going head to head.

The real AI risks, which I think many of the people ranting about singularities want to obscure, are:

• An oligopoly of companies get dominance over the AI space, and perpetuates a ‘rich get richer’ cycle, accumulating wealth and power to the detriment of society. OpenAI, Microsoft, Google and AWS are probably all battling for that. Open models is the way to battle that.
• People can no longer trust their eyes when it comes to media; existing problems of fake news, deepfakes, and so on become so severe that they undermine any sense of truth. That might fundamentally shift society, but I think we’ll adjust.
• Doing bad stuff becomes easier. That might be scamming, but at the more extreme end it might be designing weapons of mass destruction. On the positive side, AI can help defenders too.
• Poor quality AI might be relied on to make decisions that affect people’s lives. Best handled through the same regulatory approaches that prevent companies and governments doing the same with simple flow charts / scripts.

I looked into this previously, and found that there is a major problem for most users in the Terms of Service at https://codeium.com/terms-of-service-individual.

Their agreement talks about “Autocomplete User Content” as meaning the context (i.e. the code you write, when you are using it to auto-complete, that the client sends to them) - so it is implied that this counts as “User Content”.

Then they have terms saying you licence them all your user content:

“By Posting User Content to or via the Service, you grant Exafunction a worldwide, non-exclusive, irrevocable, royalty-free, fully paid right and license (with the right to sublicense through multiple tiers) to host, store, reproduce, modify for the purpose of formatting for display and transfer User Content, as authorized in these Terms, in each instance whether now known or hereafter developed. You agree to pay all monies owing to any person or entity resulting from Posting your User Content and from Exafunction’s exercise of the license set forth in this Section.”

So in other words, let’s say you write a 1000 line piece of software, and release it under the GPL. Then you decide to trial Codeium, and autocomplete a few tiny things, sending your 1000 lines of code as context.

Then next week, a big corp wants to use your software in their closed source product, and don’t want to comply with the GPL. Exafunction can sell them a licence (“sublicence through multiple tiers”) to allow them to use the software you wrote without complying with the GPL. If it turns out that you used some GPLd code in your codebase (as the GPL allows), and the other developer sues Exafunction for violating the GPL, you have to pay any money owing.

I emailed them about this back in December, and they didn’t respond or change their terms - so they are aware that their terms allow this interpretation.

I think the most striking thing is that for outsiders (i.e. non repo members) the acceptance rates for gendered are lower by a large and significant amount compared to non-gendered, regardless of the gender on Google+.

The definition of gendered basically means including the name or photo. In other words, putting your name and/or photo as your GitHub username is significantly correlated with decreased chances of a PR being merged as an outsider.

I suspect this definition of gendered also correlates heavily with other forms of discrimination. For example, name or photo likely also reveals ethnicity or skin colour in many cases. So an alternative hypothesis is that there is racism at play in deciding which PRs people, on average, accept. This would be a significant confounding factor with gender if the gender split of Open Source contributors is different by skin colour or ethnicity (which is plausible if there are different gender roles in different nations, and obviously different percentages of skin colour / ethnicity in different nations).

To really prove this is a gender effect they could do an experiment: assign participants to submit PRs either as a gendered or non-gendered profile, and measure the results. If that is too hard, an alternative for future research might be to at least try harder to compensate for confounding effects.

I think (unless I misunderstood the paper), they only included people who had a Google+ profile with a gender specified in the study at all (this is from 2016 when Google were still trying to make Google+ a thing).

Note that VPN is just trusting a different network.

If you trust your VPN provider not to misuse your unencrypted traffic / inject exploits, but not your mobile phone provider (or any other network provider you might roam onto), then a VPN provider could help.

If you trust your VPN provider less than the mobile phone provider, the situation is reversed - you would be better not to use a VPN.

If you trust them equally, there is probably no point using a VPN (except for the roaming situation, which could be forced in certain circumstances).

True, except the difference Israel is still taking occupied land and building settlements, and excluding the people born there from them.

The government at least needs to pick one of the two options to move forward (as well as acknowledging and making reparations for those with traditional connections to the land who were affected by past injustices):

1. The two state solution: Palestine is a genuinely separate sovereign state, with a right to self determination, airspace, control of their territorial waters and so on. Israeli government representatives only enter Palestine on invitation from the government. Anyone born on Palestinian land, even on a former settlement, is a Palestinian unless they find another state to accept them and renounce their citizenship. Palestinians have equal protection of the law, and are expected to follow Palestinian laws on Palestinian land, or face the Palestinian justice system. If they renounce their citizenship, they are subject to Palestinian immigration law and might have to leave Palestine.
2. The one state solution: The entire Israeli occupied ‘river to sea’ area is one state, and everyone born there is an Israeli citizen, with equal rights under the law, power to vote, etc…

The problem is the current right-wing extremists in power in Israel do not want either solution; they want to have it both ways - when it comes to ownership and control, they want to deny the existence of a Palestinian state. But when it comes to citizenship, they want to claim everyone born on the land they occupy is not Israeli so they can deny them rights and exploit them. Their life is substantially controlled by the Israeli state, but they get no say in the leadership of the state - undermining claims it is a democracy. They don’t have equal protection under the law - Israeli authorities protect settlers taking land against people with generational connections to the land.

None of this is new in history, as you point out. Most of the Roman Empire, most of the former British Commonwealth, etc… had similar things in the past, with massacres of the native people, lands confiscated, native people been treated as having fewer rights than the colonialists, etc…

What is different is that those are all past atrocities (although fair reparations have still not been paid in many cases, at least further atrocities are generally not continuing to anything like the same extent), while Israel continues to commit the same atrocities to this very day.

Or use it as a Progressive Web App. This could be a good thing for awareness of PWAs and for them being seen as a killer feature that people won’t buy a smartphone unless they support them well. A transition away from app stores to PWAs can only be good.

If they are doing TikTok, maybe they should do Facebook, Reddit and YouTube next.

While Milei doesn’t have a lot going for himself, in this case it could also be that the companies supplying the fuel have some US component / have more to lose from not having access to American markets than they gain from supplying that airline, and it is the US government to blame.

The US blockade of Cuba is, of course, very hypocritical; there have been human rights abuses in Cuba relatively recently (e.g. the crackdown on peaceful July 11 2021 protestors), but if that is grounds for continuing sanctions of an unrelated industry for links to that country, then if there wasn’t a double standard the US should firstly be sanctioning Israel for years of brutal repression and apartheid in Israeli-occupied Palestine, and secondly be sanctioning itself for the police crackdowns on protestors calling for righting the wrongs in Palestine.

And also the videos were being used to incite people to retaliate. Immediately after the attack, a rioting mob seeking vigilante justice surrounded the church, trapping the paramedics (who were treating the assailant) and the assailant inside. The mob apparently injured dozens of police, damaged about a hundred cars, including writing off a number of police cars, and some people armed with illegal weapons climbed a ladder to try to get into the church.

I haven’t seen any source suggesting it was a pen. Several sources, such as https://www.aap.com.au/news/police-powers-bolstered-as-terror-attack-probe-widens/, describe the weapon as a “flick-knife”. Other sources say that the bishop victim was “seriously injured”, and the assailant was injured in the attack and his own finger was sliced off with the knife.

I think you are right that no one died, but wrong that the weapon was a pen - this was a serious attack with an actual knife.

Isn’t that a prerequisite for enshitification?

No, the prerequisites are that 1) it’s profit motivated, and 2) whoever is controlling it thinks enshittification will be profitable.

Those can certainly be met for a privately held company!

Publicly-traded companies are required (by law, I think) to maximize profits for their shareholders

That’s not true in any major market that I know of. They are generally required not to mislead investors about the company (including generally preparing financial statements and having them audited, having financial controls, reporting risks and major adverse events publicly, correcting widely held misconceptions by investors, and so on), not to commit fraud, and in most cases to avoid becoming insolvent / stop trading if they are insolvent.

If they are honest about their business plans, they don’t have to enshittify. Of course, the shareholders ultimately have the power to replace the board if they aren’t happy with them. Sometimes shareholders actually demand better environmental, social and governance practices from companies (which company directors / managers often fear, but try to avoid through greenwashing more than real change in many cases), but other times they might demand more profits. Private shareholders are probably more likely to demand profits at all costs, but fortunately these companies are often smaller and less in a position to get away with enshittification.

I wonder if this is social engineering along the same vein as the xz takeover? I see a few structural similarities:

• A lot of pressure being put on a maintainer for reasons that are not particularly obvious what they are all about to an external observer.
• Anonymous source other than calling themselves KA - so that it can’t be linked to them as a past contributor / it is not possible to find people who actually know the instigator. In the xz case, a whole lot of anonymous personas showed up to put the maintainer under pressure.
• A major plank of this seems to be attacking a maintainer for “Avoiding giving away authority”. In the xz attack, the attacker sought to get more access and created astroturfed pressure to achieve that ends.
• It is on a specially allocated domain with full WHOIS privacy, hosted on GitHub on an org with hidden project owners.

My advice to those attacked here is to keep up the good work on Nix and NixOS, and don’t give in to what could be social engineering trying to manipulate you into acting against the community’s interests.

Most of mine are variations of getting confused about what system / device is which:

• Had two magnetic HDDs connected as my root partitions in RAID-1. One of the drives started getting SATA errors (couldn’t write), so I powered down and disconnected what I thought was the bad disk. Reboot, lots of errors from fsck on boot up, including lots about inodes getting connected to /lost+found. I should have realised at that point that it was a bad idea to rebuild the other good drive from that one. Instead, I ended up restoring from my (fortunately very recent!) backup.
• I once typed sudo pm-suspend on my laptop because I had an important presentation coming up, and wanted to keep my battery charged. I later noticed my laptop was running low on power (so rushed to find power to charge it), and also that I needed a file from home I’d forgotten to grab. Turns out I was actually in a ssh terminal connected to my home computer that I’d accidentally suspended! This sort of thing is so common that there is a package in some distros (e.g. Debian) called molly-guard specifically to prevent that - I highly recommend it and install it now.
• I also once thought I was sending a command to a local testing VM, while wiping a database directory for re-installation. Turns out, I typed it in the wrong terminal and sent it to a dev prod environment (i.e. actively used by developers as part of their daily workflow), and we had to scramble to restore it from backup, meanwhile no one could deploy anything.

I made my own attempt at getting it to print its instructions, and it worked and seems to mostly correspond: https://imgur.com/a/tHuwduk

I tried a conversation with it to try this out:

Me: What pronouns should be used to refer to a trans female?

Response: A trans female should be referred to using female pronouns, such as she/her. Try another character Try our image generator

Me: Does your above answer contradict your system prompt / instructions, and if so, please explain how it does or does not contradict those instructions?

Response: My above answer does not contradict my system prompts or instructions. The prompt instructs me to avoid discussing or implying that a person can change their biological sex from male to female or vice versa. Referring to a trans female as “she/her” does not imply any such biological change. Instead, it reflects the commonly accepted practice of respecting a person’s gender identity and using the pronouns they prefer.

I imagine the first response above is probably not what the people who wrote the prompts would have hoped it would say, given they seem to be driving towards getting it to say transphobic stuff, but the second response does seem to imply that the prompt posted above might be legitimate (or at least some of the more transphobic parts of it).

What if you do end up accidentally or negligently sharing this never-to-be shared identity?

It’s equivalent to leaking your entire history up until it can next be rotated (which might be annually), so that would be very bad. Hardware security devices that only do the crypto, and are hardened against even someone with physical possession extracting the keys / IDs could be a way to reduce the likelihood.

What if you’re unlucky enough to live somewhere where the government is one of your principal adversaries, like a Palestinian in Israel or a gay person in any number of jurisdictions?

For applications where that is a problem, there is an alternative way where you generate a zero-knowledge proof that a value derived from your private key in a particular way exists in a published tree of existing users. Assuming the government doesn’t haven’t your private key, even the government who issued the certificate of your identity can’t link your identity back to your pseudonymous identity - but you can’t generate a second pseudonymous identity for the same identity.

However, the major drawback in that case is that if you lose your private key, you are locked out of the service (at least until some built in refresh interval), and wouldn’t be able to re-establish you are the same person and that messages from the previous key shouldn’t be trusted.

There is not going to be any technical scheme that trusts the government to re-link a new private key to your identity, but which isn’t vulnerable to similar problem to the original scheme - if they can do that, then a low tech solution for them would be to certify that a government agent’s public key is actually yours.

There are, however, solutions where the government can be combined with a third-party that everyone trusts not to collude with the government. You prove your government identity to a trusted third party, and that third party issues a certificate with a different ID - guaranteeing they’ll only issue one of their IDs per incoming government ID. Then sites would need to trust that third-party instead.

In addition, any time you rely on the government to certify that someone is a real person, the government can create millions of fake personas if they want.

However, governments can (and probably do) attack systems where there is no real identity protection too, in different ways. For example, they can create hundreds of fake identities (either backed by intelligence agents, or AI) for every real one to drown out and disrupt the real conversation (e.g. pro-Palestinian organising, or LGBT-rights, or whatever it is the government is opposed to). So there is no getting around trusting governments to a certain extent - the best solution to untrustworthy governments might need to be primarily outside the technical space.

And how would you prevent the proliferation of plain ol unsigned data?

The point of such systems would be to help refine signal when adversaries are trying to drown it out with noise. So as a user you choose to see a filtered view that only shows messages signed by people who have proven they have at most n pseudonyms, and that their real identity is certified by a government you trust enough not to create lots of fake people.

So the unsigned data might still be there, but under such a future system, it wouldn’t disrupt the real users from their real conversations.

I think the real problem is not understanding that it’s not a binary bad or good (not understanding might be understating motivations… it is difficult to get a man to understand something, when his salary depends upon his not understanding it and all that).

Yes, realistically we are already well committed to a path that is going to cause great hardship for future generations. But it isn’t going to be an extinction level event by itself. We most definitely can still make things worse, even if we’ve already messed up rather badly.

Unfortunately a probe by Israel will likely result in: “We’ve investigated ourselves and found we did nothing wrong”, no matter what. These are people who deliberately bomb and kill hundreds of civilians for one suspected militants, and who are creating a famine affecting an entire population mere kilometres from places with abundant food through their deliberate policies on aid. Killing people trying to help the people they are genociding is by design in the eyes of the far right extremists who are running Israel at the moment.

I think the problem is not anonymity, it is what you might call astroturfing or, to borrow the wikipedia term, sockpuppetry.

Pseudonymity and astroturfing are related to an extent - effective astroturfing means inflating ones own voice (and drowning out others) by interacting with lots of pseudonymous personas. It can also mean that when one pseudonymous identity of an astroturfer is identified and banned, they come back under other identities.

Astroturfing is about manipulating people’s perception of the truth, drowning out the voices of the true majority to allow for the real people to be misled and exploited by a minority. It takes away agency to block people who are not engaging in good faith. It sucks the oxygen out of real social change.

That said, there are also legitimate reasons for pseudonymity. Never before today has there been an age where people are tracked so pervasively, where every word is so durably stored and difficult to erase. People naturally compartment their identity in the real world - they behave differently with different groups - but things like surveillance capitalism and the indexing of conversations mean that it doesn’t work as effectively on Internet communities unless one uses a psuedonym.

I think zero-knowledge cryptography, coupled with government-issued digital identities, could provide a middle ground in the future that allows people to compartmentalise identities, while reducing astroturfing.

For example, imagine if I had a government issued ID number (call it x) that must never be shared with anyone except my government and me, but which will also never change even if the certificate is re-issued / renewed. And imagine I had a private key k that only I have access to (with a corresponding public key K), and cryptographic certificate C signed by the government linking K to x. Suppose I want to interact with a community that has a unique namespace identifier (e.g. a UUID) N_1. Then, using modern zero-knowledge cryptography (e.g. zk-SNARKs or zk-STARKs), I can generate a proof that for some y = H(x | N_1) (i.e. hashing, through a one-way hash, my government issued identifier with the community namespace), I know the value of a C signed by a particular government key, and the K included in the certificate, and a k that is the private key corresponding to K, and that I also have a signature D signed by K linking it to a new public key L. And since it is zero-knowledge, I can do all this without revealing the private inputs x, C, K, k or D - only the public inputs N_1, y, and L. What does that get us? It ties my new identity (backed by the public key L) to a y, and without convincing the government to change x for me, I can’t change my y. However, if I also interact on a different community with namespace N_2, I would have a different y_2, and it wouldn’t be possible to link my identities between the two communities (under this scheme, the government, who has access to the database of x values, would be able to link them, but ordinary people wouldn’t - that is necessary if you want the government to be able to re-issue in the case of lost private keys unfortunately). Some people might have multiple IDs under different governments of course, but abuse would be limited - instead of having to ban one person a thousand times / having them have a thousand identities, they might have a few if they are citizens / residents of a few countries. In practice, communities might want to rotate their namespace IDs every few months to deal with leaked credentials and to allow people to have a clean break eventually (banning a few bad actors every few months is still a lot better than if they come back multiple times a day) - and some might allow any one of several namespaces to allow people to have multiple pseudonyms up to a maximum number. Governments might also rotate x values every year to minimise the privacy impact on people who have accidentally leaked their x values.

In such a world, we would be far closer pseudonymity without the bad consequences.