yep very strange. You can disconnect from Wifi to get it to work. Vscode probably keeps pinging github/microsoft before every operation.

You can also disable telemetry and that seems to work too. Settings-> search for telemetry and select "off" from the telemetry dropdown.

Function calling is a great feature. I've been using LLMs for function calls for the past few months and gpt-4 has worked great for this out of the box. Awesome to see both the models specifically trained for this.

The inference seems to be taking very long and it gets stuck at times forever (> a few minutes). My guess is that this is happening due to the load? How fast is this when it's not under load? Is it as fast as ChatGPT?

It is faster than ChatGPT normally. We're frantically scaling our infra right now.

The p50 time is around 2s end to end not under load. Bear with us as we scale our systems!

This might help: https://beta.sayhello.so/search?q=the+p50+time+is+around+2s+...

It has the code you need:

  INPUT X
  IF X > 10 THEN
      PRINT "X is greater than 10."
  ELSE
      IF X < 10 THEN
          PRINT "X is less than 10."
      ELSE
          PRINT "X is equal to 10."
      END IF
  END IF

Great! All the best scaling :) I'll give it a shot again in a bit.

Thanks! It's much faster now :)

This is amazing! Finally a use case for using all that compute power on the phone

This illustrates the beginning of use cases for computing/ML on the edge. The total power of all the phones and their sensors is mindblowing.

Some javascript crypto miners have been taking advantage of this for years now.

Time to move on to the apple bandwagon :) I've been a mac user for 10 years and it felt like Apple completely lost the plot with their laptops from 2015-2019. Glad to see them make a come back and do what they do best -- Make excellent hardware and operating systems.

Thanks for the link to the old HN item. Missed it the first time. Seems like having the ability to control the entire SoC helped apple take a lot of decisions specific to how they wanted to use the chips.

Which also implies that intel had a lot of time to catch up.

I read somewhere (possibly on this site) that intel made a purposeful decision to not invest in trying to catch up as they didn't believe in the feasibility of 5nm and lower due to difficulties managing thermal issues. Not sure how true that is but it would explain a lot.

Thanks for the links. These articles do indeed answer a lot of my questions.

The reasons company do end up with experiences like this is because they optimize for what they can measure and unfortunately user discontent with such experiences is often delayed or hard to measure.

Large companies which have a diverse user base in the hundreds of millions make decisions based on how a particular change affects the entire user population. The larger their user population, the more diverse their user base is and the harder it gets to cater to the needs of each type of user. The folks commenting here are a vocal minority, an important one given many people here are probably early adopters and also have the skills needed to build a Reddit competitor, but it's hard to see the reactions of this minority on a dashboard. A problem worth solving IMO but non trivial.

Typically initiatives like this come from a team within the company who's objective is to improve a key metric, in this case let's say user engagement. Some individual in the team probably spotted the trend that users on the app are more engaged than users on mobile web browsers. They then launched an experiment to test getting users over to the mobile app. Many users who end up on reddit via SEO probably don't know that reddit has an app and on seeing this end up downloading the app which makes them more engaged with reddit. Overall on the dashboard this shows up as a win where user engagement in the enabled group is up compared to the control and given that they give users an option to continue using the web app there is not a significant user drop. At this point, a decision needs to be made on whether to ship this change or not to ship. Folks making this decision do understand that it might be annoying to some people but the data in this case overwhelmingly supports a ship decision. They talk about it, mention their reservations, but eventually make a decision given the data and don't think about it anymore. They also don't have to feel the pain as most employees have the reddit app installed and don't see this again and again.

The key thing to know here is that there are lots of incentives in the company to make this decision a ship decision vs a no-ship decision - the data, the success of the team, the success of the individual who pioneered this change but there is not enough evidence or visible push from users to not make this change. Let's say there are some customer reports for this but unless they reach a very high volume no one is going to notice it.

Posts and discussions like this are actually a great way to get your word known to companies. This will probably stir a conversation in the team that made this change and hopefully bring out some change in the experience. Don't expect it to go away but maybe they will remember your preference of not wanting to use the app.

Would make an awesome background for chromecast.