JSON, TOML, YML? No! Apples New Config Lang!
HTML-код
- Опубликовано: 10 фев 2024
- Recorded live on twitch, GET IN
/ theprimeagen
Become a backend engineer. Its my favorite site
boot.dev/?promo=PRIMEYT
This is also the best way to support me is to support yourself becoming a better backend engineer.
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact
Kinesis Advantage 360: bit.ly/Prime-Kinesis
Hey I am sponsored by Turso, an edge database. I think they are pretty neet. Give them a try for free and if you want you can get a decent amount off (the free tier is the best (better than planetscale or any other))
turso.tech/deeznuts 
Наука
"Situation: there are 15 competing standards"
"Programmer: I want to make the end all standard that takes the best of all the standards"
"Situation: There are 16 competing standards"
That’s the beauty of standards, you have so many to choose from!
Not really. It's Apple, they can do whatever they want in their walled garden.
Jsonnet already exists too
Job security hack. Can't be replaced by AI if technology changes too quickly for AI to train!
He turned his YAML into a PKL, funniest shit I've ever seen.
Pickle rick joke is old. Old enough to know how to read and write.
@@FathDaniel 👍🥒
PISD (Post inheritance stress disorder) intensifies when someone drops DRY in middle of sentence.
I just PISD myself
you know python had Pickle-Files for years and they are nothing like this
pin this
Python pkl is still going strong, and is super useful. I was about to say Apple just don't GAF but I instantly realized: They're Apple devs. They don't reach for sensical tools to solve problems. They prob have no idea what they've done.
Was about to comment this same thing
ok I wsnt going crazy I thought i read about pickle files at some point. I thought they where just a data structure or something like pandas to manipulate and or store json data.
Pickling is just serialization of a python object. The object can in theory be anything including but not limited to JSON data.@@regfinley7111
They dropped DRY and the fear of inheritance became real.
Yeah, the main reason I'm really not keen on this even though I drown in YAML daily.
Will they also rename their brand to Apl?
They already AAPL
array programming mfs gonna flip out
They're gonna need a bigger keyboard if they want apl...
more like aplel
Array Programming Language....nah, they don't what to start that confusion. APL is one h311 of a language written by elves.
Imagine debugging your code AND YOUR CONFIG FILES
okay but imagine assuming your config file is correct when it isn't
Already do this with k8s yaml files and dockerfiles and literally all dsl files
Debugging config files isn't new, trying to find where a large yaml is wrong is hell. Debugging with help of type information and ide support for a generic non DSL config is new and spend really useful
Use Linux so you debug your workstation too
@matiasbpg to be fair jsonnet existed for a while and does kind of similar things.
One Apple to rule them all. One Apple to load them.
One Apple to compile them all, and on a white screen display them.
In the land of Macintosh where the yuppies lie.
You are so transparent, I literally see your thoughts
This feels like an excavator for your hole.
It's powerful but probably just going to dig you deeper rather than lift you up,
I found it funny that Apple is using Kotlin and Gradle to create pkl.
They are solving something that I haven't come across in my career ever and I was doing frontend, backend dev and devops with AWS and Kubernetes. It does sound interesting, until you realize that JSON schemas already exist and provide autocompletion. Also, having a build step for configs sounds like a nightmare.
> having a build step for configs sounds like a nightmare
Plus how do you configure the build step? Oh yeah there's Pkl for that...
Yeah, how would that work with Kubernetes and it's cloud secret storage/vault?
On one way, feels great. On other way, feels unnecessary. Having pkl to for checking on my toml, yaml and json config file when I try to implement a certain project is a good way to prevent me, as a User, to scratch my head when trying to check what is wrong with my deployment. On the other hand, as a dev, it can be a bad day when you need to update your project and forgot to update your pkl config because you add a new must have config variable in your python, and then you only find out about it when User try to deploy your project somewhere.
a compiled, strongly typed, nearly Turing-complete config language. is it a bad idea or is it a bad idea?
@@itssilence9998I feel the extra complexity added is the real problem. If pickle was just a standard to define the structure of your config files it would be nice since you could use it on any programming language as a library with the tools you are already using.
But no, they HAD to make it into it's own tool that requires compilation, it's Turing complete and even has a package manager. They could not for the love of god keep it simple.
EDIT:
Changed "code" to "config files", my mistake
Pickle is a well known tool for object serialization in python so naming it pickle and create a SEO mess is such an apple thing to do. Also solving a problem that doesn't exist/is solved by existing standards and claiming you done it first is also such an apple thing to do. I'm just glad I finished my pickle/unpickler in nim before this was released.
Damn, all of this to avoid XML
the funny thing is that xml has stuff like this since a long time ago, xslt.
My brother is in to some weird sadomasochism stuff. I trumped him by saying I wrote XML and xslt by hand.
@@sproccoli They said classes, I heard xs:complexType
all this to avoid ecosystems with bad tooling, by creating yet another ecosystem with bad tooling
@@ytdlgandalfget trumped. I wrote the XML/ XSLT parsers by hand.
Love the Apple hopium, can't say I'm expecting PKL to point to much better developer relations than what we've seen from Apple so far. It just makes sense for Apple to have open source buy in to config and markup systems to make their internal lives easier.
when you have a programming language to configure a program, you have a countdown until you'll need a config language to configure language to configure a program. good plan, very original, very reliable, can't wait to learn another revolutionary language, that will finally solve complexity of configuring programs.
now, seriously, has anybody had all those problems when writing a config file?
no, they haven't. this is just people overengineering something stupidly simple because they want to feel smart.
seriously, it's capitalist innovation at it's worst. instead of writing 3 awesome C libraries for parsing and validating json, toml, yaml and making bindings to 2-3 other languages, you make your own bs language, that is probably somehow tied to your platform (for security reasons, obviously) and that solves a problem that nobody asked to solve.
there has to be "a choice" of syscalls, basic filesystems, rendering mechanics and ui libraries, so that one wouldn't be able to just cross-compile an app from windows to posix and back. same as there has to be a choice of socket voltages and phone chargers. isn't it nice to "have options"?
@@Daniel_Zhu_a6f Yeah, just write your JSON config validation logic in C, lol.
@@ra2enjoyer708 validation can't be harder than parsing. and people write far more complex parsers in C than a json one.
but maybe you should just use python scripts for that.
@@ra2enjoyer708 i mean, they could have written them for you, so that you wouldn't have to. most languages have c interface. it's strange that i have to mention this, really, but most parsers and numeric libs are implemented in C/C++/Fortran anyway.
4:29 What a subtle way for Apple to say "Stop kidding with all these different languages and specific DSL and just do everything in *OUR* environment." ;p
this reminds me of something that used to be in the odin standard library. It was a configuration language that allowed you to refer to things in diffrerent parts of the config and apply relational/arithmetic operations to it. last time i looked it wasn't there anymore, but it was pretty cool.
I'm with prime on DRY. Everyone's obsession with this concept means they couple it with premature optimisation and you never find the real patterns and it becomes a clusterfuck fast.
Is this just apple-branded dhall?
Edit: yes it's just dhall, but Apple
Well who knows dhall? Now everyone knows pkl
My god, I've been in the game for a long time. This is the first time I've heard of D-Hall. We have too many of these things. Thanks for teaching me something new.
Google's internal deployment configuration files used a python-derived DSL called PCL. The programming element was essential to resolving the final command line used to start a binary within a borglet. The previous configuration syntax, borgconf, didn't allow as much expressiveness and ended up with a lot of cases that couldn't be addressed.
22:34 I loved the reaction from the flash bang LOL.
To me that has the vibes of 15th competing standards.... I am also not quite big fan of making simple things be dynamic and more complex that they need to be.
I see two main reasons to like/want this:
1. DRY your hart out with the config
2. Type hinting
Both are weak souse excuse to make a whole new config lang/standard.
There are already options to have minimal DRY abilities when doing config, having something that goes ballistic with abstraction will makes configs way harder to work with and maintain.
Type hinting seems like a great lure but really now if the config is simple, readable and not abstract we need less type validation in general, the more we abstract the more we need validation and what good is validating that I entered a string and not a an integer when the string was a database name and the string I entered is still the wrong name...
How do we go about making the validations and abstractions safe and not allow you to shoot yourself in the foot, should we also have testing framework and write tests for our config?
that moment when you throw the flashbang and it hits a corner before coming back to you
22:40 the best part
Nearly unalived himself and everyone watching.
I thought I was going to make it through a day without having to learn a new syntax.
one issue I have with this is that you still need to define env variables somewhere. that means having most likely a .env.template which I this point I would just use for all configuration variables. I wrote my own lib in Kotlin to create those template envs from kotlin and I think that's already more than enough. Kinda unsure if I wanna duplicate the work
Great they recreated Groovy. Now give them all credits
Groovy when strings. 10 ways, none work good without Jenkins complaining
He addressed why using Typescript is a bad idea, which also applies to Groovy. I've seem some pretty ridiculous "configuration" files that were basically application code
I’m oncall today and that slack sound gave me a panic attack lol 😂
YAML is an absolute nightmare to work with. I don't understand why so many products and tools use it.
Because JSON was a nightmare to work with. Especially so, since you can't add comments.
Yaml is fine, what is ur issue with it?
I've been writing a lot of docker compose configs and it has been great
Although I prefer toml
It is because YAML is the most expressive, writable and readable of all of the static markup syntaxes.
@@arjix8738 "ur" argument is based on the most trivial usage of yaml. Yaml becomes non-trivial when "ur" manifest files either have to span multiple screens or has to be divided into multiple files due to the sheer size of the config, like when you deal with e.g. nested Helm charts or kustomize for business critical stuff.
Looks cool. The language bindings to generate the DTOs definitions for an API spec is a lot like using protoc and protobuf but without imposing a protocol, only the types that are expected.
It will impose the types, since you are only be able to operate on types understandable by Pkl and which can be mapped to your target language type. So you'd still have to handwrite extra transform step in the language of your choice which transforms lists in the config into `LazyStreamableOrderedUniqueSet`'s.
@@ra2enjoyer708 you are right. The same is happening to me right now with sqlc, lol. Anyhow because DTOs come at json or other serialized form, it is reasonable that there is mismatch of types with the model layer so to speak
This is more like less advanced version of the dhall programming language than nix. Anyway my opinion is that configuration should be done in a general programming language that can be used for configuration (lua, scheme, python...) instead of a configuration language that can be used to program. It's even better if the configuration language is the same as the app language, this can be practical also in compiled languages if you have a fast enough compiler and hot-reloading.
Yes - it's funny seeing all these configuration helpers when the Lisps solved that problem before it even existed.
Somehow this reminds me of a build system called Bazel. Where you have a language called Starlark, which is annoyingly similar to Python (except when it suddenly isn't). The end result is you have to create a ton of Python-y build definitions and whatnots and... well, we had 15 competing standards...
i tried to build a rather popular open source project that uses bazel, could never grok what the fuck i'm supposed to configure to get shit right. i quit trying to contribute right then and there
New PasKaL just dropped
I've had many situations where I've wanted this, but its already too easy to add a thousand hyperspecialized libraries to your project, so I'll likely avoid it at all cost.
this will be great for deployers and operators and is super useful for validation, i've lost countless hours just sifting through helm chart values.yml files and deploying wrong configuration. developers will have to however push validation and maintenance of config outside of their applications to this which can be both good and bad
I love how your head gets translucent.
DRY = Do repeat yourself
anytime I'm writing a program that I know only I will ever configure, I just use a raw text file or environment variables, and anytime I'm writing something that other people will have to look at, I go with either json or ini because they're familiar
Dude I never run into ini files in the wild
@@KManAboutyou may run into toml which is basically ini++
Love seeing prime’s hair becoming one with the background
I like that yaml and json do not allow programming functionality. Once you have the potential for conditions and flow control now we have to think about how this gets tested since it's actually code.
So… Dhall but worse?
My first thought, exactly.
I probably won't use this in the production builds of applications, but being able to define a schema and automatically generate code to parse it, and in the future also generate man pages and cli arg parsing/help, would be incredibly useful.
Are you my professor of operating systems by chance
@@ci6516 no, I'm a student.
so XSD which we have fore ages...
Neovim will be default.. Only for the small price of $699
But, you can upgrade from 8G of RAM to 16 for the low, low price of $299
@@adammontgomery7980 CPU not included. xD
...a month
What if u generate code that generates code?
Like python to cython to c to assembly.
Cue already exists
Cue is all the greatness of this but without the inheritance footgun + a super neat scripting layer which I think the Primeagen would love.
Unfortunately Cue doesn't have the resources of Apple behind them so the language is still working on an LSP and package manager. Once those are ready to go, I'm all in on Cue.
Well let me Cue you in. Cue isn't queued up for use.
So how is this better than XSD, JSON schemas or whatever variation? Basically push XSD in LSP to get autocompletion in editor?
Everyone keeps forgetting of Nix
14:48 In Hungary we explicitly have 4-digit zip-codes :)
"Your age will never be a long int"
Oh my Lord
"It won't even be a short"
Oh Jesus
this is beautiful, as someone who makes configurable software to be used in prod/dev and we don't want to expose values and environment stats, this allows us to migrate and control data that is also easy enough that anyone can step in and reconfigure without things breaking. imagine if all config files used this.
and I get your point on extraction, it introduces a layer of complexity and dissassociative on debugging, my counter for this is config files. they should be a dynamic form that can't be broken or extracted from a formula. else you may as well use typescript for your configs. I honestly love everything i see in Pkl and I am creating something similar for an admin interface.
Non-programmers creating code configs.
wild right?
but its because we have an event system that fires micro instances of objects on a per trigger event. and these need to be configured per managments needs. so i've made my own mutable pkl like parser.
can you even pickle an apple?? 😂
Apple:
"I got seeds too, can ya pkl me too Greg?"
I mean Cue has been there all this time, and is much simpler, no OOP nonsense in there
wow, now I can enjoy Stored Procedures in data files...
While the regex for zip code is always a mistake for user input, my understanding is that this is there to catch errors by people doing the config, not to sanitize malicious input.
Question to you: how do you spell this one - data or data?
Will try it to manage large openapi yaml specs
i love this. this will make writing kubernetes manifests so much faster
14:48 there are placed with alpha-numeric zip codes too.
like the UK, which was also mentioned in the video
I'm using handlebars templating engine in rust to generate configs
Light theme incident is golden! :)
You can use references in JSON and YAML so you don’t need to repeat everything. But it lacks flexibility in cases where strict „copy and paste“ isn’t enough.
So, this is like Nix, but not nix?
Yeah, except Nix can do way more because of the package manager.
I guess that can be seen as dead weight if all you wanted was a config language..
This is not really like Nix. More like cuelang.
This is similar to nix with types. The closest to Pkl I know is dhall.
@@Saeppel jsonnet
Or Nickel? (a potentiellement successor of Nix)
Xcode has vim motions, but it's missing some functionality
Bruh my eyes 23:00, Also your hair look funny af!
I fell from my chair laughing when Prime did 🤣🤣🤣
Can't believe I cheeked out my slack😂😂😂
So it is like Nix except for the infinite recursion errors.
Skill issue
Jsonnet isn't bad but pkl seems a bit more tightly defined.
Jsonnet works well, but its very loose, and you would want an external validate so it's cool that you get that in the tool.
Jsonnet is a little heavy as well...
22:33 light mode is kryptonite for hooded developers 🙂
Lua was basically this. Legit, lua started life as a programming language for configuration and is still used that way. e.g neovim
It's puh-kewl actually!
A use case I’m thinking of is ci. Let the company write scary yml configs that could potentially break prod. But have a small set of experts in pkl write some ci to validate that the yaml they wrote passes the pkl file. Is that something that’s possible?
How does this compare to cue?
Pkl has type inheritance meaning values can be overwritten at any point. The Turing completeness without the immutability is a setup for a bad time in my opinion.
22:37 Nice hair
In order to be considered a "true" programming language you need to be able to compute any computable function - i.e. it needs to be "Turing complete'. Prolog doesn't have explicit branching or loops, however it is Turing complete.
time to debug the config that generates the code that generates the config that generates the helm values that generates the K8s manifest that generates the deployment logic that generates the node and network calls that end up breaking because of the UI entered config your IT department put in when setting up network drives that back up your node.
It’s going to be great.
if programming languages are defined by turing completeness, then apparently some dependently-typed languages like agda or coq cease to be languages because they are not turing complete. they are used to write formal proofs, and as such, any well-typed program must be guaranteed to terminate on all inputs, which instantly means that they are not turing complete, because you can have turing machines that never terminate on some inputs.
My first thought was actually “configuration doesn’t start with a K…” 😅
That RegExp matches any string that contains 5 digits in a row 🤣
Yep, its a bad rule. They suck.
I love that his hair is now partially transparent xD
Are you about to move to Mac and use Yabai for window management?
the only things required to be turing complete are 1: ability to read and write memory in some way (file text counts) 2: the ability to move around the memory, and 3: the ability to determine which of the above to do at any given time.
This reads a lot like a quote from Doctor Who. "First you have to imagine a very big box fitting inside a very small box. Then, you have to make one. It's the second part people normally get stuck on."
Lol, I also immediately thought is this just nix lang? Though I'll give you nix is maybe a bit to functional for some people and can be slightly hard to read at first. Also nix doesn't really have typed properties though I hope it gets them in the future.
Nix won't have types in any time, but Nickel is being built to maybe replace Nix some day
Blue haired agen: have you heard of cuelang? Seems similar to pkl
17:01 How much cares is put into that variable?
6:00 what do you mean? That's Lua.
Clojure's EDN is all you need.
Not everyone likthes lithsp
@@KManAbout yeah, not everybody has good taste
the apple bit off the pickle from python
Link?
Why note json schema / schema store was the first thing that came to mind
That's what I came to ask.
How fast apple will put it under paywall?
GIF Jif, Char Car things can be both wrong and right at the same time
I'm going to call it π-call
23:00 oscar performance
I think they will be really soon flooded with requests for more validation options. It will be really hard to draw the line between validation and a logic around the data.
IMO data and logic should not be overlapping. You could roll your own CI/CD using a Python script that consumes a JSON data structure and some env variables (or values from a secret store) and it would be better than using YAML.
Apple lets their employees often code in whatever IDE or framework they want to. I am sure that varies team to team but its not rare.
How quickly Bun can add it? :D
28:00 that commenters argument doesn't really make sense if you're just restricting and validating data for a configuration not moving program logic to a config.
Otherwise data validation in databases would break program logic or something.