Location: Cambodia (UTC+7)

Remote: Yes

Willing to relocate: Potentially

Technologies: Python/TypeScript of late, C/C++, C#, Java, Scala, JS, Kotlin, Android, More

Résumé/CV: LinkedIn (https://www.linkedin.com/in/niqd/details/experience/), GitHub (github.com/sublimator)

Email: ndudfield@gmail.com

Curious debugger, doer, and hands-on maker of things, creative and self-reliant.

Ideal for early-stage companies. Worked remotely since 2011 from Cambodia for U.S. companies, including SublimeHQ/Ripple/Coil and smaller NFC/RFID startups. Easily managed 12hr timezone differences, and open to working within EU time zones for even better alignment.

Like many, I am presently interested in AI(esp. browser++), however I remain open to anything interesting.

Thanks.

Super Kind folk, and Brave too. I once offered to build something for free for them to prove I could build things, but they said there was no need and kindly paid me. I didn't get the gig in the end, but in this cold age of frauds, I really appreciated that.

"YET ANOTHER shiny new toy to distract me. Can't help myself even though I think it's mostly a waste of time"

Am I just projecting? Relatable, in any case :)

I immediately implemented streaming into my rocketchat gpt bot, was definitely a distraction but my colleagues liked it. No more waiting until the complete response is sent.

Yep, you captured the moment ^_^

  Location: Cambodia (UTC+7)
  Remote: Yes
  Willing to relocate: Potentially
  Technologies: TypeScript of late, C/C++, C#, Java, Scala, JS, Kotlin, Android, Python, More
  Résumé/CV: LinkedIn (linkedin.com/in/nicholas-dudfield-3b938629), GitHub (github.com/sublimator)
  Email: ndudfield@gmail.com

Curious debugger, doer, and hands-on maker of things, creative and self-reliant. Ideal for early-stage companies. Worked remotely since 2011 from Cambodia for U.S. companies, including SublimeHQ/Ripple/Coil and smaller NFC/RFID startups. Easily managed 12hr timezone differences, and open to working within EU time zones for even better alignment.

Like many, I am presently interested in AI(esp. browser++), however I remain open to anything interesting.

Thanks.

It seems "local" is all you need :)

Interesting! What's the prompt eval processing speed like compared to llama.cpp and kin?

I haven't run any specific low level benchmarks, lately. But chunked prefilling and tvm auto-tuned Metal kernels from mlc-llm seemed to make a big differenced, the last time I checked. Also, compared to stock mlc-llm, I use a newer version of metal (3.0) and have a few modifications to make models have a slightly smaller memory and disk footprint, also slightly faster execution. Because unlike the mlc-llm folks, I only care about compatibility with Apple platforms. They support so much more than that in their upstream project.

thanks, I'll give it a crack

...how did you highlight a specific sentence like that?

Looks like a Chromium-specific feature: https://web.dev/articles/text-fragments

Pretty cool. Doesn't work on Firefox.

It just worked on Safari on iOS. That’s pretty impressive.

https://caniuse.com/url-scroll-to-text-fragment -- yes Safari supports it

nexusraven/starling-lm/chat.lmsys.org, all from UC Berkeley and affiliates

"Chatbot Arena is an open-source research project developed by members from LMSYS and UC Berkeley SkyLab"

Something in the water there?

OpenAI also had years "flying under the radar" training on API requests

Don't think it adds a lot of value. One, the instruction-following models do not create high quality q&a pairs, chats do it by default. Two, at the time the usage was very low compared to ChatGPT. I am fairly confident, one month of people trying all sorts of things w ChatGPT gave them a lot more data - both in quantity and range - compared to API usage previously. Third, they always maintained they do not train on API inputs, and I think it's fair to assume that. They do not get feedback whether the output was any good, so it's always dicey to feed that directly to a model without those signals.

They don't train on API requests

Why should anybody believe them?

Segment ids are all you need (probably)