Zos_Kia

I’ve only had issues with fitgirl repacks i think there’s an optimisation they use for low RAM machines that doesn’t play well with proton

Interestingly the pendulum is now swinging the other way. If you look at next.js for example, server generated multi page applications are back on the menu!

I’d place it right around when angular started gaining traction. That’s when it became common to serve just one page and have all the navigation happen in JavaScript.

Honestly the use case i’m working on is pretty mind blowing. User records an unstructured voice note like “i am out of item 12, also prices of items 13 & 15 is down to 4 dollars 99, also shipping for all items above 1kg is now 3 dollars 99” and the LLM will search the database for items >1kg (using tool calling) then generate a JSON representing the changes to be made. We use that JSON to make a simple UI where the user can review the changes - then voilà it’s sent to the backend which persists the change in database. In the ideal case the user never even pulls up the virtual keyboard on their phone, it’s just “talk, check, click, done”.

That’s fucking badass thanks for the pointer this might prove useful. In the structured output department i’m hearing great things about dotTxt’s outlines which lets you constrain output according to a regex, but i haven’t tested it yet.

I’m currently a guy working on something like this ! It’s even simpler as you can have structured output on the chatgpt API. Basically you give it a JSON schema and it’s guaranteed to respond with JSON that validates against that schema. Spent a couple weeks hacking at it and i’m positively impressed, I have had clean JSON 100% of the time, and the data extraction is pretty reliable too.

The tooling is actually reaching a sweet spot right now where it makes sense to integrate LLMs in production code (if the use case makes sense and you haven’t just shoe-horned it in for the hype).

If I understand these things correctly, the context window only affects how much text the model can “keep in mind” at any one time. It should not affect task performance outside of this factor.

Yeh, i did some looking up in the meantime and indeed you’re gonna have a context size issue. That’s why it’s only summarizing the last few thousand characters of the text, that’s the size of its attention.

There are some models fine-tuned to 8K tokens context window, some even to 16K like this Mistral brew. If you have a GPU with 8G of VRAM you should be able to run it, using one of the quantized versions (Q4 or Q5 should be fine). Summarizing should still be reasonably good.

If 16k isn’t enough for you then that’s probably not something you can perform locally. However you can still run a larger model privately in the cloud. Hugging face for example allows you to rent GPUs by the minute and run inference on them, it should just net you a few dollars. As far as i know this approach should still be compatible with Open WebUI.

There are not that many use cases where fine tuning a local model will yield significantly better task performance.

My advice would be to choose a model with a large context window and just throw in the prompt the whole text you want summarized (which is basically what a rag would do anyway).

LGTM ⛴️

If you like to write, I find that story boarding with stable diffusion is definitely an improvement. The quality of the images is what it is, but they can help you map out scenes and locations, and spot visual details and cues to include in your writing.

What’s hilarious about this is that they are so sensitive to being described. As a filthy liberal, you can call me weird is be like hell yeah strange-pilled bizarre-maxxing. Call me a communist I’ll be like you know it brother, stalin-with-laser-eyes.gif.

But call a right winger a weird fascist and they’re whining “well now let’s not resort to insults”. Why can’t they tolerate what they tend towards? Doesn’t sound very empowered to me.

holy shit you’re right i don’t know where i got the idea that it was the same format

To clarify : We’re talking about differences in the codebase here. They are still exactly the same game, with some very minor disparities in certain mechanics.

The technical differences tend to disappear over time because they rely more and more on the datapack format, which is shared between the two codebases.

But we are talking about freelancers, not about SEO or content marketing, more like content filling

Most SEO is done by freelancers (at least in my industry). When i talk about content marketing i mean anybody who writes blog posts and LinkedIn posts for companies. It was already shit long before AI arrived.

Yeah I’m not bashing anybody, my wife did that for a couple years I know how it is. There was a kind of golden period where it would even pay enough to let you do some quality stuff but when VC money stopped raining the market slumped almost immediately.

I think the bitter lesson here is that there’s a bunch of jobs where quality has zero importance.

If you take for example, content marketing, SEO, and ad copy writing… It’s a lot of bullshit, and it’s been filling the web with gpt-grade slop for 20 years now. If you can do the same for cheap I don’t see a reason not to.

I’m so glad they ditch the mediocre discourse on AI, God, and the creation of mankind in favor of balls to the wall horror. We’ll see how it goes!

This exact joke is used in a Community episode, but I never saw it attributed to a professor

Even without seeders, you can sometimes be lucky and resurrect old torrents that have been kept in cache by providers such as real debrid