• 1 Post
  • 42 Comments
Joined 3 years ago
cake
Cake day: October 4th, 2023

help-circle



  • As some comments here have pointed out, it’s probably more like data that goes into the training corpus.

    I’d also suggest that some demand won’t be from AI companies, but rather from other consumers of data storage, even if a boom in AI-driven spending is at the root of things. Like, maybe your ideal storage medium is SSD, but AI-related demand has driven prices up to 5x what they were earlier. Okay, so some users get priced out and fall back to rotational drives, because those are good enough for their use case. Prices on those go up by 2x. Well, the next fallback for some of those users who get priced out, who maybe used hard drives to store backups, say, is gonna be magnetic tape storage.



  • “A lot of people want local inference,” he said. “And somehow, they assume, that the problem is solved now. Because we are going to have open-source models. We’re going to be able to run them on our own computers. And so we’re not going to need all the stuff in the data center.”

    “But from a global allocation perspective, actually, it’s quite terrible to have everybody have a bunch of the world’s limited supply of [dynamic RAM] sitting in their own computer,” he added.

    Yeah, I’ve made that point here many times myself. Capacity utilization is going to be much higher in the cloud, so it’s cheaper to do in the cloud than to have everyone with their own inference hardware.

    But…that depends on the constraint being limited on hardware. That’s the case in 2026, but unless much more hardware-intensive models keep showing up and being much better than came before, so that someone like OpenAI can get value that increases tremendously out of more hardware for a model, that hardware constraint won’t persist.

    Local compute also provides some benefits. You don’t rely on Internet connectivity, so you don’t have things go down if your network connection does. While we don’t have many latency-critical applications out there today, you can get better latency. Unless and until OpenAI is going to provide service with some pretty hard and convincing data privacy guarantees, you get data privacy. Whether it’s a company that might be exposing their core IP or what their people are working on, an individual who doesn’t want their sexy chatbot sessions showing up places they don’t want them one day, a government who has national security concerns, or anyone who is just worried about a cloud LLM provider leveraging lock-in down the line to extract concessions or more money for them, local LLMs provide a lot of benefits.

    I am confident that the world as a whole won’t shift to local AI for at least four or five years, and probably more. It can’t—most memory right now is going to cloud AI, and reduced capacity utilization means that getting the same compute capacity would require more memory than is out there, we don’t have construction for many times existing memory capacity underway yet, and it would take four to five years for that to happen if we started today.

    But I do think that it will tend to happen over time.

    I think that in the long run, if cloud AI companies are going to persistently keep people on remote compute, it’ll need to be because their models stay private and their performance is so much better than open-weight models that people choose them for quality, not for hardware scarcity.


  • My guess—and I’m not privy to any inside information here—is that operations at places that are building neural-net-based models look something like this:

    1. “Harvest” people are responsible for obtaining data and as much associated information as possible, stuff like where data was obtained from, timestamps, any identity information, etc (handle “X” on platform “Y”). If a comment or webpage or whatever gets revised, maybe they keep multiple copies. They’ll avoid things like following infinite loops on websites, deal with corrupt archives, all that. That probably goes into some giant database. Maybe some of these guys buy access, like Google did with Reddit. A lot are just going to write spiders, scraping bots, whatever. A search engine fundamentally has to have a spider slurp down data to create its index to work at all, so I’m sure that Google is going to have a bunch of expertise in this area, built up over many years. They’ll probably retain this data in pretty raw, unprocessed form.

    2. “Unification” people are going to work on the above. They write code to convert the above into a standardized-across-all-platforms format. Maybe they convert stuff to unify timezones or time formats on comments. Maybe they strip off formatting stuff added by particular platforms. Markdown or HTML titles, or titles in Microsoft Word documents, for example, probably wind up represented in one way. Where possible, maybe usernames get attached, if data is from a forum.

    3. “Scoring” people work on the unified dataset. They are going to be developing some system—and there might be a number of parallel projects—for generating a score of various sorts that might be a useful value for either a model or training information. With this as well, search engines have to do this; Google’s PageRank, for example, tries to assess the “value” of a page, determine what’s most-useful and reliable information, so that it can order results (and maybe blacklist some). A lot of what search engines do isn’t just getting you hits, but putting the most-valuable ones at the top. If you can develop heuristics to rank information based on reliability or political position or any number of other things, those are going to be useful numbers.

    4. “Pruning” people use information, probably including the “scoring” information, to winnow down the size of the data, eliminate data where the cost of inclusion in the training set—it’s costly to train a neural net, and reducing the size of the training corpus reduces that cost—outweighs the benefits. They probably try to cut out garbage, irrelevant binary data, poisoned data, duplicate data, misinformation, whatever.

    5. “Training” people are going to do the training and probably develop systems for rating the value of the above “scores” in producing desirable output. Maybe some “scoring” heuristic is really super-useful in producing accurate and useful output as defined by some benchmark that these guys set up. Like maybe, oh, one “score” is based on whether information was obtained from a .gov TLD, or another whether a text classifier says that the language is English and the Lix school year readability is above a certain level. Could be many of these. Some of what these guys find may go back to the “scoring” guys in trying to guess what heuristics should be further developed (“known time of domain age to time content was initially spidered is particularly valuable in identifying SEO-generated spam website content; see if you can produce more scores that identify mass-created spam websites”), some to the “pruning” guys to increase the low-value data that they prune out, and some of what they find may go into weighting importance of text for the training process. Some may even go all the way back to the “harvest” guys to determine how frequently to spider data from a given source—Google doesn’t respider information at a fixed rate, but hits “valuable” sources, like major news sites, very frequently. For non-pure-LLM neural nets—I don’t know if Google is doing this yet with Gemini, but I’d imagine that eventually, probably more-advanced AIs will need to move past just being an LLM—this might also be an input used in making a decision at runtime.

    There are probably going to be a lot of points in this process where not-valuable data gets identified and removed or deprioritized. Even if no engineer at somewhere like Google has tried to write a “scoring” heuristic to identify text specifically to identify output from a tool that mass-edits comments on Reddit, my guess is that text consisting of nonsense words, which is what some of the Reddit mass-comment-editing-tools I’ve seen insert, probably intrinsically get identified as part of that process by multiple classification tools that generate heuristic scores. If all of a user’s comments look like the output of some simple text generation tool, or if all of them change in a given short period, that’s probably kind of a big red flag. That does mean that the information won’t be in the output model, but I think that people who are aiming to poison LLMs during training probably are expecting more impact from putting nonsense in their comment versus just blanking the comments than they’ll likely have. Google—and more broadly, people who are running search engines and even more, building neural nets trained on data on the Internet—are just going to have to already have a lot of fairly-sophisticated work in place to assess the value of and filter their inputs.


  • I can no longer create a new one without using an email tied to my identity

    I haven’t bothered to look into it, but historically, there were a lot of sites that let one create throwaway email addresses to avoid handing over email addresses and tying an online handle to personal information.

    Sites that harvested this email data fought back by trying to blacklist those “throwaway email address” sites, but it’s a game of whack-a-mole—like, those throwaway email address sites also set up and discard domains from a pool—so I imagine that it’s likely possible to manage to find one that works, if you’re determined to do so.

    Note that throwaway sites do create security issues in that password recovery and similar often relies on a registered email address, so you’re potentially entrusting some random throwaway email site with access to your account, since it can receive email from the site. I wouldn’t use that for something important, but depending upon use, it could be useful.

    Also, a number of sites—I don’t know about what Reddit will be doing, but I think that Google requires this for account registration these days—require a phone number, and at least in the US, while you can get VoIP SIP service, don’t need to tie that to a phone, I understand that there have been legal restrictions added on SMS service from this crowd in the past few years. I don’t know if Google permits for validating a phone number via voice or requires demonstrating SMS capability.


  • I’d imagine that that probably increases the magnitude of the effect I mentioned in another news article that I linked to here in another comment, where Google users are just using Google’s Gemini LLM rather than searching for information on Reddit. Google will already have incorporated that information into its training corpus for Gemini back when it paid Reddit for access to it, so it makes the LLM the only route for a user to get access to that knowledge.

    (Not criticizing your move, just commenting on what I imagine the broader impact will be, particularly if the “logged-in only” move increases the rate of other people wiping their posts and comments as well.)



  • While referral traffic is probably mostly people using Reddit as a store of knowledge, doing things like site:reddit.com searches on Google, not principally use-Reddit-as-a-forum users, one does imagine that trying to require users to use Reddit logged-in will probably cause more users to leave Reddit. This is probably an opportunity for the Threadiverse to pick up refugee users from Reddit. Might be good to simplify and smooth the path for new Threadiverse users as best as possible, since I imagine this’ll probably be the last “big moment” with potential friction for Reddit users in getting Reddit users over the hump to using a walled garden.

    Reddit will start testing the login requirement in “select,” non-US markets in Q4. The company will use those tests to decide what a logged-in-only Reddit will look like “globally,” per Huffman.

    It won’t be all at once, then; Reddit will probably want to assess what how severe the impact on userbase is, how many people they can convert to using their app or having a login, and what tactics will work best in converting those users. So one might expect influx of new Threadiverse users over time as Reddit gradually shifts regions to being login-only.


  • “During October, we have seen that search traffic has dropped by more than 1 million referrals on average per week. If this rate of decline continues, current trends would suggest that total DAUqs [or daily active unique users] could fall in Q4, by a low-double-digit percentage from our current base of 130 million DAUqs,” Huffman wrote.

    Huffman would like Reddit to become less dependent on search referrals. “Logged-in and app DAUq are our most valuable users, and they are our primary focus,” he wrote.

    https://www.reuters.com/business/reddit-pinterest-shares-tumble-weak-user-revenue-growth-2026-10-30/

    Reddit plunges as Google AI search squeezes traffic

    Oct 30 (Reuters) - Shares of Reddit (RDDT.N), tumbled 23% in premarket trading on Friday after it warned Google’s growing use of AI in ​search was hurting referral traffic and could shrink its user ‌base, offering fresh evidence of AI-driven disruption.

    • Reddit’s warning highlights how Google’s ​shift toward AI-powered search, which can answer queries directly, is ⁠disrupting the flow of traffic to websites that have traditionally relied ​on the search engine for users
    • Total daily active unique visitors could fall ​by a low double-digit percentage in the fourth quarter from about 130 million if referral traffic continues to fall, Reddit said


  • I can’t speak as to all areas, but I’m in the San Francisco Bay Area, which is pretty wealthy and progressive, and if anything, I’d call people in the area generally upbeat relative to people I’ve run into elsewhere. Certainly nothing like the kind of “everything is awful or probably will be, doom doom doom” stuff that I’d say I see a disproportionate level of on the Threadiverse.

    I have wondered before whether maybe it’s correlation with some of the userbase.

    • I’d say that there are probably more furries on here than I run into elsewhere, but I don’t really notice the pawb.social home instance being attached to downbeat comments.

    • Ditto for Linux users, but I’ve been on a lot of Linux-oriented online forums, and that doesn’t normally seem to be a particularly depressed or negative crowd elsewhere.

    • The Threadiverse is more European than most groups I’ve run into, and maybe I could buy some correlation there—there is correlation between less sunlight and depression, and Europe is home to the world’s biggest concentration of people who get less sunlight—but I’m confident that the continent as a whole can’t be this negative. I also spent a lot of time on /r/Europe prior to moving to the Threadiverse, and it was nothing like this.

    • There’s a higher concentration of far-left users than anywhere else I’ve been. Maybe I could buy that. Like, if your life is heavily-focused on wanting The Revolution to come and private industry dissolved and all that, okay, the world, which isn’t doing that, is maybe a disappointing place to be in.

    • I’m pretty sure that there’s a higher concentration of transexual users than on any other group that I’ve spent a lot of time around. Maybe if you figure that a disproportionately high percentage of trans people are unhappy with their body, that could really be weighing on people, making them unhappy. Or maybe from medication…hormones can affect mood, and maybe hormone replacement treatment can cause negative moods?

      searches

      It sounds like maybe HRT can be a factor in depression: https://www.reddit.com/r/MtF/comments/9g68zm/is_depression_a_possible_side_effect_of_hrt/

    EDIT: I have seen articles that show correlation between social media use in general and depression, and obviously everyone here is a social media user. Maybe that could explain some difference between real-world groups, but that doesn’t explain the discrepancy with the many other online forums I’ve been on.


  • I am suspicious that there is a disproportionately-high-to-society-at-large ratio of people who have clinical depression on here.

    I’ve never run into as many people in any group, in real life or in online communities, who show up and persistently take bleak outlooks on damned near everything. It’s not just “the glass is half empty”, but “the glass is half empty and probably what water is there is contaminated and it’s going to dry up and if I drink it I’ll probably choke to death and…”


  • To take it a step further, I work adjacent to OEM manufacturers for industrial control systems. Half of our customers are running windows 98.

    Hah! I’d actually consider industrial automation to be one of the more-likely applications for LLMs. Not necessarily LLMs driving the system, but outputting software to drive PLCs and similar. My understanding from past reading is that industrial automation is one of the principal things that China is interested in using LLMs for.

    searches

    https://faxiangongchang.com/en/reports/china-industrial-ai-agent-2026

    What AI Agents in Factories Are Actually Doing

    Down on the shop floor, industrial AI agents are currently running most solidly in four types of scenarios.

    Quality inspection. Vision large models are replacing traditional rule-based machine vision. Among the 12 lighthouse factories newly added by the World Economic Forum in September 2025, AI and generative AI use cases account for up to 50%, with demonstration factories reporting results including a 41% reduction in product defect rates.

    Process parameter optimization. In the Baosteel hot-rolling line case disclosed by Huawei Cloud, the Huawei Pangu large model improved finishing-mill width-spread prediction accuracy by more than 5% and raised yield by 0.5%, which by its own account translates into more than 90 million yuan in additional annual revenue. It should be noted that such figures are all vendor-disclosed and still lack independent third-party verification, but directionally they are consistent with public data from lighthouse factories.

    Engineering and maintenance assistance. Siemens Industrial Copilot improves the efficiency of generating programmable logic controller code by roughly 60%, and pilots of its maintenance edition show an average 25% saving in reactive maintenance time; the product won the Hermes Award at Hannover Messe 2025. The factory operations agent Microsoft launched at Hannover Messe 2025 lets employees ask in natural language, “Why did Line 3 stop yesterday?”

    Scheduling and order management. This is the entry point closest to AI agents for small and mid-sized factories. An AI tool aimed at small and mid-sized factories has disclosed a set of figures: after adopting AI scheduling and order tracking, factories’ average order delay rate fell from 25% to within 5%; one factory in Dongguan cut its order-splitting preparation time by 60%; and an annual fee of about 10,000 yuan corresponded to a return of roughly 1:12. These are again vendor figures, but they touch on an important fact—the value of AI agents for small and mid-sized factories lies first not in sophisticated process optimization, but in freeing up the labor tied down by high-frequency, tedious work such as order tracking and scheduling.

    They mention PLC code generation, but I guess some other areas too.


  • Not to say that it couldn’t be an annoyance in terms of bandwidth contention, but I’d be a lot more concerned about what’s uploaded by smart speakers than what’s downloaded.

    If we’re talking bandwidth contention, as far as I’m aware, traffic-shaping functionality in routers for downstream traffic isn’t great. We have good functionality for prioritizing and limiting traffic on the sending side of the bottleneck link. That side knows how congested the link is. With most Internet connections, though, the last-mile link is the bottleneck. That means that effective shaping is gonna have to happen on the ISP’s side (and the ISP doesn’t know how the consumer wants prioritization done, and there’s presently no mechanism for consumers to inform the ISP), or have smarter routers on the consumer side that can make themselves the bottleneck and shape there. The latter would be nice, as one could solve ISP bufferbloat on the consumer side.

    I would think that it should be technically possible for a home router to passively detect upstream link saturation by monitoring traffic and then using ECN or packet drops to preferentially throttle a lower-priority connection.

    Linux monitors RTT average time for TCP connections, which one can see in ss -i output. So the endpoint kernel is going to have enough information to passively infer link saturation on the downstream side via looking for correlated RTT increases. The problem is that a given machine on a network doesn’t know what other machines are doing, so it’s not in a great position to infer link saturation or shape its own traffic. You’d want the router to handle it.

    I don’t know if NAT is typically implemented by having a router proxy TCP connections or whether the NAT table is lighter than that, works mostly at the packet level.

    checks an OpenWRT box doing NAT

    No entries in ss or netstat output, so I would guess not.

    Still, even if the Linux kernel on a router isn’t presently tracking that statistic, it’d be possible to do so.

    Once you have downstream bandwidth, you have something like a rate-limited, fair, prioritized qdisc on a Linux router with the rate kept just below the point where RTTs start spiking.



  • I mean, why wouldn’t they? Like, say you’re making, oh, a deathmatch shooter. Cheating kinda ruins gameplay for players. If the competitive multiplayer crowd winds up on console, then…shrugs that’s where you do game development.

    The PC world really pioneered a lot of competitive (remote) multiplayer game genres because PCs with network connections were widespread before consoles with network connections were. And to some extent, genres adapted to PC hardware rather than console hardware. High-speed twitch mouse+keyboard—PC hardware—first-person shooters dominated in the late '90s and early 2000s. That’s what players who could play the games had. MOBAs are often (unless things have changed recently) adapted to the mouse.

    But I don’t think that there’s any fundamental link between PC hardware and competitive multiplayer games. You could make games adapted to console hardware just as well. And if it’s more important to prioritize a level playing field and avoiding cheating for competitive multiplayer games…I mean, sure, I’d focus on console releases too.

    The flip side of that is that I think that in general, the PC is a better platform than consoles for non-multiplayer-competitive games, and if those don’t make it to consoles…shrugs I mean, same thing, just cutting the other direction.


  • I’ve made this comment before, but to reiterate, I think that consoles are generally a better solution for competitive multiplayer games.

    Consoles were designed from the ground up to be locked down. PCs were designed to be open and modifiable. Trying to figure out ways to jury-rig a PC to act locked-down is always gonna be an uphill battle.

    Also, since this is for competitive multiplayer games, consoles also provide a level playing field on hardware. No pay-to-win “buy this hardware to get an edge” stuff. There’s a whole industry of PC “gamer” hardware sold using basically “pay to win” marketing.

    PCs are nice because they’re open. That lets you make one faster or prettier or use any input device or rig up macro software or mod games, fiddle with memory or whatever. And that’s a good thing for games that aren’t competitive multiplayer games. You can play the game however you want, and it’s not like it negatively affects anyone else. But that same characteristic that’s an advantage for non-competitive-multiplayer gaming is a drawback for competitive multiplayer gaming.

    For competitive multiplayer gaming, you want a platform that provides equal footing for all players as much as possible and is hard to modify, and that’s what consoles do well.