given your experience i believe using azure web apps with an api gateway will get you a good runway. you can deploy to Azure directly from github actions and configure api gateway along with Azure AD to handle auth and security.
If you're already using Azure, they offer Linux stuff too. You can move away from Windows without moving away from Microsoft (if you want to). Or you can find equivalent services on AWS or Gcloud, or more abstracted ones like Vercel or Netlify (who whitelabel and value-add features & ease of use on top of big clouds).
i'm using azure, aws and gcp at work and personal projects. i recommend azure for better management UX and relatively transparent cost and billing management. once you setup your infrastructure as code you can move anything but the vendor specific services easily. Azure AD is easy to migrate from, API gateway not so much.
is the tipping thing correct? I provided the same prompt to ChatGPT and received multiple emojis without offering a tip.
prompt: you're Ronald McDonald. respond with emojis. what do you do for fun?
answer::circus_tent::hamburger::juggling::party_popper::balloon::game_die::french_fries::performing_arts::rolling_on_the_floor_laughing::people_holding_hands::rainbow::art_palette:
It's also non-deterministic if you drop the temperature to zero. The only way to get deterministic responses is to lock the seed argument to a fixed value.
TLDR: Developers can now specify seed parameter in the Chat Completion request for consistent completions. We always include a system_fingerprint in the response that helps developers understand changes in our system that will affect determinism.
Thank you, I should have been more specific. I guess what I’m asking is, how deterministic would you say it is in your experience? Can this be used for classifying purposes where the values should not be outside what’s given in a variable input prompt , or when we say deterministic are we saying that , if given the same prompt then the output would be the exact same only? Or is the seed a starting parameter that effectively corners the LLM to a specific starting point only then depending on the variable prompts, potentially give non-deterministic answers?
Perhaps I’m misunderstanding how the seed is used in this context. If you have any examples of how you use it in real world context then that would be appreciated.
I’ve not had any success to make responses deterministic with these settings. I’m even beginning to suspect historic conversations via API are used to influence future responses, so I’m not sure if it’ll truly be possible.
The most success I’ve had for classifying purposes so far is using function calling and a hack-solution of making a new object for each data point you want to classify for the schema open AI wants. Then an inner prop that is static to place the value. Then within the description of that object is just a generic “choose from these values only: {CATEGORIES}”. Placing your value choices in all capital letters seems lock it in to the LLM that it should not deviate outside those choices.
For my purposes it seems to do quite well but at the cost of token inputs to classify single elements in a screenplay where I’m trying to identify the difference between various elements in a scene and a script. I’m sending the whole scene text with the extracted elements (which have been extracted by regex already due to the existing structure but not classed yet) and asking to classify each element based on a few categories. But then there becomes another question of accuracy.
For sentence or paragraph analysis that might look like the ugliest, and horrendous looking “{blockOfText}” = {type: object, properties: {sentimentAnalysis: {type: string, description: “only choose from {CATEGORIES}”}}. Which is unfortunately not the best looking way but it works.
The point is you do not have a valid counterexample since you are using a different workflow than what's described in the article.
In my personal experience working with more complex prompts with more specific constraints/rules, adding the incentive in the system prompt has got it to behave much better. I am not cargo-culting: it's all qualitative in the end.
You can usually just say something like: "You must respond with at least five emojis".
Sure, there are cute and clever ways to get it to do things, but it's trained on natural language and instructions, so you can usually just ask it to do the thing you want. If that doesn't work, try stating it more explicitly: "You MUST... "
hey! i really like this. it's a great idea and implemented neatly. my MacOS arm install is <100mb. i know some asked for tabs. i would like to be able to open multiple windows.
parent company got hit by cyberattack yesterday. salute to all IT and InfoSec colleagues working around the clock while everyone else is taking the day off.
Thanks. By default, BoltAI stores user's API key in Apple Keychain. So other applications and scripts won't be able to access this key without triggering the OS's security dialog.
Secondly, all requests are sent directly to OpenAI servers. There is absolutely no middle/proxy server.
There is however a proxy server for Claude but this was due to a technical difficulty and user can host their own proxy server instead.
It is not a perpetual license. "One year of free updates included." means subscription. Especially considering product depends on third party API, which rapidly evolves, you cannot really stick with the old version, because some new model will not be embedded in the one year old version of the app you have. So, in reality you pay $40-80 a year for this product. Plus your ChatGPT API Key usage. In that case I would just suggest pay for the OpenAI.
If you are looking for an alternative of the app for the ChatGPT - I would suggest https://www.typingmind.com I have been using it since it was released. It is also available with Setapp subscription, which gives you some free usage of the OpenAI with Setapp embedded API key.
Technically it's perpetual fallback license, which was popularized by the Jetbrain team [0]
> A perpetual fallback license is a license that allows you to use a specific version of software without an active subscription for it. The license also includes all bugfix updates, more specifically in X.Y.Z version all Z releases are included.
After 1 year, you can continue to use the app without having to upgrade. You won't receive new features, but will continue to receive bug fixes for the features released previously.
So I can argue it's not "subscription".
Plus, renewal cost 50% less, so it's not $40-$80 a year, but only $20-$40 a year after the first year.
And lastly, the app fetch model lists from OpenAI API so there is no need to "support new models".
I believe this is a sustainable pricing model for an indie developer like me. And I believe it's fair for everyone.
I feel like if you have to specify “technically”, you agree with me that it is basically a subscription.
As an Indie developer myself I do not support subscription based licensing (or your version of license) for B2C products. You can offer B2B subscription based licensing, but B2C should always be perpetual licenses. Sure it is ok to ask for the upgrade to new version, which has significant features.
And to clarify, we are talking about the product that does not have any outgoing server spending.
I feel like Sublime Text showed the best of it. You purchase the new license every new major version.
Affinity apps are the same. Acorn. Alfred. And a lot more.
There are certainly some apps that release a major update every year. But in that case you pay for work already done. In case if you pay for subscription, you are going to see minor updates “bug fixes”.
I don’t see a reason for an indie developer to charge subscription, if there is no server cost. New versions should just make the app interesting for more users, so the income should be flowing constantly every month.
Anyway, I just personally don’t support subscription based apps and never will.
