Well, you have basically implemented a Java type-system level type checker for SQL. I don't believe there is any type system strong enough to express the whole thing without the escape hatches (casts), besides Lean, Coq and alia.
Not sure what you mean. I meant that without declaration site variance, ("pragmatic") unsafe casting is everywhere in jOOQ's internals. Without being able to capture wildcards in local type variables, ("pragmatic") rawtypes are everywhere as well (check Collections.swap() for an illustration)
jOOQ doesn't get involved in any such prefetch/eager fetch shenanigans, which are often wrong. They work for some cases and do too little/too much for others. So, specifying the exact data to fetch in every query is a much better approach, ideally with nested collections:
https://blog.jooq.org/jooq-3-15s-new-multiset-operator-will-...
> Not sure how successful jOOQ has been financially, but considering they've been around for many years at this point, I have to imagine it's worked out well enough to pay for the lights and kibble?
Nice talk, you're a great speaker! And BTW I also love jOOQ :)
Your example translated to Permazen Kotlin (for concision) would be something like this:
data class IncomeByDate(val film: Film, val date: LocalDate, val income: Long)
films
.flatMap { film -> film.rentals.map { IncomeByDate(film, it.date, it.amount) } }
.groupingBy { it.film }
.reduce { _, l, r -> l.copy(income = l.income + r.income) }
.values
.sortedWith(comparing<IncomeByDate, String> { it.film.title }.thenBy { it.date })
That's not using a convenience library like your jOOL.
You can argue against this in a bunch of ways. We could debate readability, the fact that it's not Java (doesn't matter technically) or that maybe the RDBMS parallelizes operations. We could add parallelism with a .parallelStream() in the right spot easily enough. But the query is around the same length as the SQL and reads in a similar fashion.
You discuss this a bit later but then say, look how much time we've spent! Sure, it comes later in your talk, but that doesn't equal time spent. You're comparing a SQL query you presented fait accompli, vs half a talk of iterating on the Java.
I think you probably overestimate how easy SQL is because you're an expert in it. For occasional users like me, it can be a quirky pain. Even the way strings work is unintuitive. But we know the standard libraries of our languages pretty well, we have to, it's required for the job. Your whole product is built on the fact that SQL isn't good enough, there's a lot of problems that remain unsolved when you just use SQL. Otherwise jOOQ wouldn't exist.
That's before we get into the different properties of the backends, e.g. horizontal read/write scalability for free (FDB) vs RDBMS, incremental online schema evolution and so on.
I don't know if your various flatMap / etc methods are purely implemented in the client (it would be quite bad from a performance perspective? But since you're implementing reducers in kotlin, I guess that's what this is), or if you somehow translate the AST to SQL (similar to jinq.org in Java or Slick in Scala or LINQ in .NET).
But in either case, I think that mimicking "idiomatic" client APIs is more of a distraction than something useful. I've explored this here, where I was asked about my opinion on Kotlin's Exposed:
https://www.youtube.com/watch?v=6Ji9yKnZ3D8
Obviously, this is ultimately a matter of taste, but just like all these "better SQL languages" (e.g. PRQL) come and go, these translations to "better APIs" also come and go. SQL is the only thing to stay (has been for more than 50 years now!)
> We could add parallelism with a .parallelStream() in the right spot easily enough
You typically don't even need to hint it, the optimiser might choose to parallelise on its own, or not, depending on production load... Anyway, that's an overrated topic, IMO.
> I think you probably overestimate how easy SQL is because you're an expert in it.
I'm happy when coding in any language / paradigm. When working with XML, I will happily use XSLT, XPath, etc. When working with JSON, I don't mind going into JavaScript. I'm just trying to stay curious.
I really don't think that SQL is "harder" than any other language. It may just be something certain people don't really like, for various reasons.
> Your whole product is built on the fact that SQL isn't good enough
I think you're projecting your own distaste into my work here. I love SQL. SQL is wonderful. It's quirky, yes, but what isn't. jOOQ users just don't like working with an external DSL within Java (though many are very happy to write views and stored procedures with SQL and procedural extensions). There's no need for a false dichotomy here. I've worked on large systems that were mostly implemented with SQL written in views, and it was perfect!
Also, jOOQ is much more than just the DSL. SQL transformation, parsing, etc., it's a vast product. The string-y SQL folks could use it as a template engine, without touching the DSL, and still profit from parsing / transformations / mapping:
https://blog.jooq.org/using-java-13-text-blocks-for-plain-sq...
Some customers use jOOQ merely to migrate off Oracle to PostgreSQL (via the translating JDBC proxy).
And I'm looking forward to the OpenJDK's Project Babylon. Perhaps we'll get actual macros in Java, soon, which could work well with jOOQ.
Anyway, I didn't mean to hi-jack too much. It's great when people try out new / different approaches. I'm just triggered whenever someone claims that SQL is harder than anything else, when they should have said, they prefer other things and don't want to learn more SQL (which is fine, but quite a different statement).
Permazen is a library that runs in-process, there is no network protocol, so it expects to be close to your data. When you use flatMap then yes that's compiled for a for loop under the hood, and reads on the objects trigger KV reads. There are ways to use hinting and pre-fetching and such if latency starts getting higher, say if you use FoundationDB (which does have a protocol). But in most KV stores even those over the network, there is a local cache and you can pre-read keys close to each other in keyspace.
I guess by easy/hard, I am considering experience to be 99% of that. If you transform XML by using XSLT then that's great, but most devs won't know what to make of that, because they lack the experience. I haven't used XSLT since, I think, 2001, and if I had to use it today I'd need to relearn it from scratch. That's not hard hard, it's just time consuming, and I'd rather write perhaps slightly more verbose or less pretty code in a language that me+team already use than (re)learn a DSL in that case.
> I don't know if your various flatMap / etc methods are purely implemented in the client (it would be quite bad from a performance perspective?
Haven't quite groked how it works myself; the readme does call out that if your data is separated from your application by a high latency network then performance likely won't be good enough.
I'm interested in taking a look at this as I maintain apps that use local kv stores so this could complement the approach I'm already using.
The "object oriented" in jOOQ Object Oriented Querying stands for an object oriented query model, not mapping to object oriented target data structures.
