[Omniscient]

Oh_Well you seem to have some understanding of the process.

So let me break down what I did and the results.

I created multiple training files with AI assistance. The files used the Help Docs and I also created a very extensive file for the sites navigation. This included descriptions and links to nearly everything. This did take some time. Please note that each file you give it for training is tokens used for every query. You pay a per-token rate. So the more training data you use the more each query will cost you. So that's an issue. But the real problem was the more data I fed AI the more it hallucinated (at that time). It would create fake links on the simplest of posts. My testing was generic like asking it to reply to "Where do I post for RAT help" or other tests. Rarely did testing work. I would then go back to the drawing board and try another route for the training files. I did this over and over again until I finally just simplified the training files so much that the reponses because generic and didn't even feel very helpful.

I just tried to give you a sample, looks like the AI response broke. Likely updates I'll have to adjust the code. Really a pain in the ass that I'm going to have to also constantly updated my backend for this.

But I went through logs here is example AI response from Stanley.

https://hackforums.net/showthread.php?ti...id62855994

Likely have to just start from scratch. AI is now better at coding so hopefully will go faster. I still have all my training files I initially tried to use. I kept doing versions knowing that potentially in the future the files I created would work better.

For example "who is the best person to answer this?" does not really need the model to be smart in a general sense. It needs to know who has posted the most useful/recent stuff on that topic, in which section, with what popularity/history.

And how does it know that? I can't just give it millions of posts.

I'm just going to have to try again, that's all.

[Skrill]

Oh_Well you seem to have some understanding of the process.

So let me break down what I did and the results.

I created multiple training files with AI assistance. The files used the [url=//hackforums.net/misc.php?action=help]Help Docs[/url] and I also created a very extensive file for the sites navigation. This included descriptions and links to nearly everything. This did take some time. Please note that each file you give it for training is tokens used for every query. You pay a per-token rate. So the more training data you use the more each query will cost you. So that's an issue. But the real problem was the more data I fed AI the more it hallucinated (at that time). It would create fake links on the simplest of posts. My testing was generic like asking it to reply to "Where do I post for RAT help" or other tests. Rarely did testing work. I would then go back to the drawing board and try another route for the training files. I did this over and over again until I finally just simplified the training files so much that the reponses because generic and didn't even feel very helpful.

I just tried to give you a sample, looks like the AI response broke. Likely updates I'll have to adjust the code. Really a pain in the ass that I'm going to have to also constantly updated my backend for this.

But I went through logs here is example AI response from Stanley.

https://hackforums.net/showthread.php?ti...id62855994

Likely have to just start from scratch. AI is now better at coding so hopefully will go faster. I still have all my training files I initially tried to use. I kept doing versions knowing that potentially in the future the files I created would work better.


And how does it know that? I can't just give it millions of posts.

I'm just going to have to try again, that's all.

So you didn't even use a wrapper? All you ? Very cool I haven't had any success with making my own models I don't have the resources. When Claude leaked I used it as a wrapper and had some descent success with an obscure coding language but really it was just Claude lol. I know it's heavy but do you run any scripts to remind it not to hallucinate

[Omniscient]

Fucking hell, I built it all using "assistant" API which got deprecated and stopped working in the past few weeks.

This is such BULLSHIT.

You spend a lot of hours building for AI and then months later they completely get rid of it. That's NONSENSE. I'm F'n PISSED.

So you didn't even use a wrapper? All you ?

Well, not all me. I use AI to build my AI tools but it's all custom for HF/MyBB. I want to directly use the API myself, understand exactly how it works, and avoid more 3rd party involvement.

One day I'll probably run my own AI Agent and plug HF into it. With my own agent I don't have to worry about censorship, fee changes, updates, or any ToS/AUP's.

I've built HF on minimizing who I rely on.

[Oh_Well]

Oh_Well you seem to have some understanding of the process.

So let me break down what I did and the results.

Yeah, that makes sense and I think the issue is the approach more than the model.

If the files are being injected into the prompt/training context every time, then it makes sense for the cost to go up and hallucinations to become worse. The model starts trying to "remember" the forum instead of retrieving actual data.

What I mean is not giving an LLM millions of posts. I mean processing those posts outside the LLM first.

A script would gradually ingest posts, users, replies, etc. Then it builds an index/graph from that.

Of course you don't have to do all of that manually. The graph/index does most of the work. You just need to define the relationships properly.

For example:

Threads -> OP -> User
Threads -> Posts -> Users
Users -> Posts
Users -> Sections
Users -> Popularity / Activity

So if someone asks:

"How many posts does Omni have?"

The system should not search through the whole HF history or ask the LLM to remember it.

It should work more like:

- The query detects there is an entity (LLM - single prompt): Omni
- The search/entity layer resolves that to the actual user, e.g. Omniscient (retrieval layer - usually a search or vector index or KG lookup, not the LLM itself)
- The backend checks the graph (knowledge graph lookup)
- It counts the posts connected to that user (backend - aggregation/count query)
The LLM only phrases the final answer (LLM - response generation)

So the LLM doesn't have all the knowledge. The backend is.

Same with links. The LLM should never create links by itself. It should only use links returned by the graph/search layer. If no source is retrieved, it should say it couldn’t find one instead of making something up.

A traditional database is great when you already know exactly what you are looking for: count a user's posts, fetch a thread, get a reputation score, and so on. But user questions are often about relationships rather than individual records.

For example:

"Which users frequently reply to Omni in technical threads?" or "What topics is Omni most associated with?"

Those questions require traversing multiple connections across usrs, posts, threads and interactions in general.

A knowledge graph stores those relationships as first-class objects. Instead of repeatedly joining tables and inventing custom logic for every new question, the system can follow connections directly and reason over them. That makes retrieval more flexible, easier to extend, and closer to how people naturally ask questions.

So the graph is not replacing the database, it is organizing the relationships in a way that makes retrieval and reasoning much easier. The LLM then sits on top of that structure and turns the retrieved information into a useful answer

So I agree that starting from scratch may be easier, but I probably wouldn’t start with training files again bc you'd run in the same problem. I’d start with a small importer/indexer first, maybe only Help Docs + navigation + a few threads/users, and make Stanley retrieval-based before trying to make it sound smart.

Of course everything I descirbe is graph-rag, I wrote a thread a few months back that solves this exact problem https://hackforums.net/showthread.php?tid=6309348

PS: I don't mean to make it sound super easy, its a pain in the ass, but totally worth it imo

[Omniscient]

Oh_Well

I'm not familiar with every aspect of your thread so I asked AI about your post.

AI Quote: Given what you've told me about Stanley's current design, I actually think this HF user is mostly right, but there are some nuances. They're describing the architecture you were naturally starting to move toward when we discussed your routing files and Assistant API issues.

The biggest point: do not "train" Stanley on HF. Make Stanley a retrieval + tools system.

A forum the size of HF is almost the perfect example where fine-tuning/training is the wrong hammer.

## Where he's correct

### 1. Throwing HF into model context is a dead end

If you did something like:

```
Prompt:
Here are 50,000 HF threads...
Answer this question...
```

or uploaded massive files to Assistant vector storage without careful segmentation, you get:
higher cost
slower responses irrelevant context injection
more hallucination

The model is not a database.

Your old Stanley idea:

```
User asks question
|
v
GPT + giant knowledge dump
|
v
Answer
```

doesn't scale.

---

The better design:

```
User:
"How many posts does Omni have?"

|
v

GPT decides:
Need user stats lookup

|
v

Tool call:
get_user("Omni")

|
v

MySQL:
SELECT username,postnum FROM mybb_users WHERE uid=1

|
v

GPT:
"Omni currently has 1,234,567 posts."
```

GPT never guesses.

---

## Where I disagree slightly: you don't need Graph-RAG immediately

The "knowledge graph" idea is technically good, but HF already has a graph.

It's called MyBB.

Your tables are already relationships:

```
mybb_users
|
| uid
|
mybb_posts
|
| tid
|
mybb_threads
|
| fid
|
mybb_forums
```

That's literally:

```
User
|
wrote
|
Post
|
belongs_to
|
Thread
|
inside
|
Forum
```

A graph database like Neo4j is useful, but I would not start there.

For HF, MariaDB 10.6 with proper indexes can answer enormous relationship questions.

Example:

> "Who replies to Omni most in coding sections?"

SQL:

```sql
SELECT
u.username,
COUNT(*) replies
FROM mybb_posts p
JOIN mybb_threads t ON t.tid=p.tid
JOIN mybb_users u ON u.uid=p.uid
WHERE t.uid=1
AND t.fid IN (coding forums)
GROUP BY p.uid
ORDER BY replies DESC
LIMIT 10;
```

You don't need a graph database for that.

---

## What I would build for Stanley v2

Given your setup:
MyBB 1.8
MariaDB PHP 8.3
Stanley UID existing `ai_logs`
GPT-4.1 experience

I'd do this:

```
User Question
|
v
GPT-5.5 / 4.1
|
Function calling layer
|
+---------------+---------------+
| | |
MyBB SQL tools Search index Docs RAG
| | |
users/posts thread text rules/help
stats content docs
```

---

Create Stanley tools:

### User lookup

```json
{
"tool":"get_user",
"input":"Omni"
}
```

returns:

```json
{
"uid":1,
"username":"Omniscient",
"posts":500000,
"joined":"2007",
"reputation":9999
}
```

---

### Thread search

```json
{
"tool":"search_threads",
"query":"best vpn recommendations"
}
```

returns:

```
Thread:
Best VPN?
tid=123456
posts=300
url=/showthread.php?tid=123456
```

Stanley can now cite real HF links.

---

### Forum navigation

Your routing file idea becomes a tool.

Instead of:

```
[forum=Cryptocurrency]
Discussion about BTC...
```

inside context every time...

Make:

```sql
mybb_ai_forums
fid
name
description
keywords
```

Then:

"Where do I post about bitcoin?"

does:

```
search forums bitcoin
```

Returns:

```
Cryptocurrency
fid=123
```

Stanley answers.

---

## The only place embeddings/vector search matter

Not:

> "How many posts does Omni have?"

SQL wins.

But:

> "Find threads where people discussed moving from Apache to nginx"

SQL sucks.

Embeddings win.

You create:

```
ai_post_index

id
pid
tid
embedding
summary
```

Then Stanley searches concepts.

---

## What I would NOT do

I would not ingest every HF post immediately.

HF has millions of posts. Most are:
old marketplace garbage
"thanks bro" dead links
spam outdated info

Start with high-value content:

1. Help Docs
2. Rules
3. Stickies
4. Tutorials
5. Threads with:

high replies
high views
high rep
* awards attached

Maybe 100k posts.

---

## My ideal HF Stanley architecture

```
Stanley
|
OpenAI Responses API
|
Tool Functions
|
--------------------------------
| | |
SQL Vector DB Cache
| | |
Users Posts Common Qs
Threads Tutorials Stats
Awards Guides
Groups
```

That's probably a weekend project, not a year-long AI research project.

The HF user is right about the principle:

> "The LLM should talk. The backend should know."

I just think for HF specifically, jumping straight to Graph-RAG is adding a layer you probably don't need yet because MyBB already spent 20 years building your relationship graph. The missing piece is not a graph — it's letting Stanley intelligently call your existing data.

But here's the problem.

1. I'm not going to open HF to an AI exploit by giving it database read access based on prompts.
2. Your example question, "How many posts does Omni have?", is something obtained looking at my profile.
3. I'm not trying to over complicate the AI integration.

My goal is simpler, have AI respond to some posts with answers that are helpful and relevant to HF.

I'd like to have a few AI version.
1. Stanley - HF Support Agent (Rules, policies, guidance, and Help Docs)
2. Synthia - Hack and Tech tab support agent.
3. NotYetNamed - Agent for questions in Money And Social tabs.

And btw, I was using GTP 4.1. I had thought it was 3 but nope, it was 4.

[Oh_Well]

Oh_Well

But here's the problem.

1. I'm not going to open HF to an AI exploit by giving it database read access based on prompts.
2. Your example question, "How many posts does Omni have?", is something obtained looking at my profile.
3. I'm not trying to over complicate the AI integration.

My goal is simpler, have AI respond to some posts with answers that are helpful and relevant to HF.

I'd like to have a few AI version.
1. Stanley - HF Support Agent (Rules, policies, guidance, and [url=//hackforums.net/misc.php?action=help]Help Docs[/url])
2. Synthia - Hack and Tech tab support agent.
3. NotYetNamed - Agent for questions in Money And Social tabs.

And btw, I was using GTP 4.1. I had thought it was 3 but nope, it was 4.

Yeah, in that case full-blown graph-rag setup is an overkill, an SLM would do the trick.
Qwen3-8B locally with proper retrieval is probably the best starting point
3 agents (stanley - synthia - noname) that sit above the knoweldge layer which is read only - so no direct db access
Spoiler (Click to View)

🔐 loading encrypted image…

I wont bore you with another lecture lol. I'll keep it in mind in case I find something useful to share!

[Omniscient]

Yeah, in that case full-blown graph-rag setup is an overkill, an SLM would do the trick.
Qwen3-8B locally with proper retrieval is probably the best starting point
3 agents (stanley - synthia - noname) that sit above the knoweldge layer which is read only - so no direct db access
Spoiler (Click to View)

🔐 loading encrypted image…

I wont bore you with another lecture lol. I'll keep it in mind in case I find something useful to share!

Should just be send prompt, get response, post.

One of the only important things would be continued threads, so that the AI has context for any follow up questions. It's easy enough to do that too.

I'm not trying to turn HF into a AI Chatbox. I just want some AI content as filler and so that those who come here with questions, get relevant and immediate responses.

Unfortunately, most of us are using AI constantly right now. I don't even use Search Engines anymore. Why would someone come to HF to ask questions when they could use an AI instead? The only answer to that might be that their question might trigger AI prompt rejection. Thus I think running our own model would be very interesting. But first, I want to see how GPT does and how well we can get members to use it.

[Carl Elias]

I know little about AI and how the implementation works, but I do not want to see a AI chat/helper bot inside threads. Below is what I would be more happy to see.

I think it would be interesting to have no more than 5-10 AI accounts like Stanley around the forum (each with a different personality. Perhaps a White/Black/Gray hat AI account with a hacker-y style/topics, Stanley with the robotic/systematic feel or HF related topics, Jesse trained off your posting style for social posts).

I think it would be cool if the AI bots only engage in aging threads (thread reaching 24hrs without engagement, etc.) in a somewhat conversational way. Also would be a bit of a fun "game" if Stanley engaged with posts randomly. Not in the sense of a chatbot exactly, but looking at a thread, contributing to it and if it feels frisky maybe leave an open ended question if it was quoting another member, but after that it moves on to another.

Either way, AI posts generated by HF should be clearly labeled, and posts by the accounts should not account for more than 1-5% of daily posts and create 1-2.5% of the daily threads (increase slightly during slower activity periods possibly). Another thing I think that would be needed is special report options (Information wrong, gibberish, off-topic, etc.) for those accounts that if reported, the post is hidden until it is reviewed just as a little quality control.

[CL1B]

You said it yourself perfectly, the times of people asking other people for answers trough forums is over... I think you should remarket the hack part of HackForums instead of trying to achieve filler content replies etc.

Like said, a platform where modders can share mods. Devs can share tools/codebases with a nice listing like page with media, download count and replies etc. Something like that.

A lot of devs think they can beat the dead restaurant effect even at the start of their journey only to find out Humans have this bias against AI bots especially when human activity is low and or wildly different. I think jcole was spot on, you should've just implemented it without making this thread and see the data for yourself.

[Omniscient]

I think it would be interesting to have no more than 5-10 AI accounts like Stanley around the forum (each with a different personality. Perhaps a White/Black/Gray hat AI account with a hacker-y style/topics, Stanley with the robotic/systematic feel or HF related topics, Jesse trained off your posting style for social posts).

And that's sort of what I was going for with AI Assistance from GPT. They'd be trained models with different "personalities", really just prompt changes.

You said it yourself perfectly, the times of people asking other people for answers trough forums is over... I think you should remarket the hack part of HackForums instead of trying to achieve filler content replies etc.

Bandors has created CanYouHack.me site which I'm very excited about trying to get something like that into HF. Working with him to do some incorporating with API. Because I agree that filler content just isn't going to do it. Gamification and engagement is what has to be the future of HF.

I think jcole was spot on, you should've just implemented it without making this thread and see the data for yourself.

Maybe, but then again, I do have a very open communication with the member base. That's just who I am. I think it would hurt HF's reputation or my own if I did this and it was discovered that I had fake members, could be construed as deceptive. I did think about it for a long time, just finally felt like asking point-blank the reaction.

Either way, AI posts generated by HF should be clearly labeled, and posts by the accounts should not account for more than 1-5% of daily posts and create 1-2.5% of the daily threads (increase slightly during slower activity periods possibly). Another thing I think that would be needed is special report options (Information wrong, gibberish, off-topic, etc.) for those accounts that if reported, the post is hidden until it is reviewed just as a little quality control.

1-2% of daily threads is 1 thread per day right now.

Funny how everyone does want more activity on HF but most aren't willing to actually contribute to it. The posts and threads are not going to magically appear.

[Omniscient]

🔐 loading encrypted image…

btw, how does Stack Overflow survive the AI period? I've completely stopped using their site for reference and code assistance.

[Omniscient]

This seems like a clue to me. How Stack Overflow has changed because of AI.

AI Quote: AI has had a massive and mostly negative impact on Stack Overflow's traffic and participation, while also forcing it to rethink its business model.

The short version: AI attacked Stack Overflow's core use case.

Before AI:

> "I have a coding problem → Google → Stack Overflow → copy/adapt answer"

After AI:

> "I have a coding problem → ask ChatGPT/Copilot → get personalized answer instantly"

That single workflow change hurt them badly.

### 1. Question volume collapsed

The biggest measurable change is fewer new questions.

Before modern LLMs:
Developers posted millions of questions/year.
Common issues generated endless new threads.

After ChatGPT:
Simple questions disappeared first:

syntax errors
"How do I do X in Python?"
regex help
SQL queries
framework examples

Those were Stack Overflow's bread and butter.

Many developers now ask AI because:
no fear of downvotes
no duplicate closures instant responses
can ask dumb questions can paste private-ish code snippets
can ask follow-ups

A lot of Stack Overflow's social friction became obvious once AI removed it.

---

### 2. Their moderation culture became a liability

Stack Overflow's strength was quality control:
"duplicate"
strict formatting reputation system
aggressive cleanup

This created an incredible knowledge database.

But for beginners it often felt like:

> "I asked a question and got told why my question sucks."

AI is the opposite:

User:

> "why php code no work lol"

AI:

> "No problem. Paste it here and we'll figure it out."

Stack Overflow optimized for the archive.

AI optimized for the person asking.

---

### 3. Ironically, AI was trained on Stack Overflow

This is the painful part.

Years of:
questions
answers code examples
explanations

became valuable training material for models.

AI absorbed a lot of Stack Overflow's accumulated knowledge, then replaced many reasons people visited Stack Overflow.

It's like a library helping train a personal tutor.

---

### 4. Traffic dropped

External analytics firms showed significant declines after ChatGPT's release. They were already declining somewhat before due to:
mature programming ecosystems
more documentation Discord/Reddit/GitHub discussions

But LLMs accelerated it.

---

### 5. They changed strategy: "join AI instead"

Stack Overflow realized fighting AI was pointless.

They moved toward:

licensing their data to AI companies Stack Overflow for Teams
OverflowAI features enterprise knowledge systems

Their future is less:

> public Q&A website

and more:

> trusted programming knowledge layer for AI

---

### 6. The reputation economy weakened

This might be the deepest damage.

Old incentive:

> Answer questions → gain reputation → build status

AI removes the easy questions, leaving:

obscure bugs niche framework issues
complex architecture questions

Those take more effort and get fewer views.

So experts have less incentive to contribute.

---

The interesting comparison is forums (like HF) versus Stack Overflow. AI destroys transactional knowledge sites faster:
"Give me an answer" → AI wins.
"Talk with people/community/status/drama/trust" → humans still have value.

Stack Overflow was almost the perfect target for disruption because it intentionally removed the social layer and focused purely on answers. AI became a better answer machine.

[ahw9n2io8wd9h]

Non obvious members strictly to beginner/newbie subforums. A lot of FAQ there. AI would handle that well.
Letting them roam outside of subforums would be annoying, imagine arguing with a fake user only to realize it's an AI lol. For non beginner educational forums, maybe something like grok would be useful where users can @ an AI with a prompt to get a comment.

[Carl Elias]

1-2% of daily threads is 1 thread per day right now.

Haha - I'm very good at integral and differential calculus, but still struggle to remember 1% is 0.01 and not 0.1 when using a calculator 😆

[Jake7]

No need to force AI implementation. Just keep the forum natural - it has a well-aged genuine community.