Further, VPR's algorithms tend not to be designed with performance in mind; simulated annealing as a placement method does not scale well past the tens of thousands of LUTs, which is why nextpnr moved from full simulated annealing to [heterogenous analytic placement](https://ieeexplore.ieee.org/document/6339278), and recently we have been working on a [multi-electrostatic placer](https://dl.acm.org/doi/abs/10.1145/3489517.3530568); both significantly more scalable methods based on using mathematical optimisation methods to place things nearby.
let me take this opportunity to ask a question about something that makes absolutely zero sense to me: why did yosys insist on sticking with tcl over some more modern/well-known/familiar scripting language? yes everyone has their favorite tcl scripts for vivado or intel or whatever but none of them are transferrable to yosys irrespective of that fact that yosys supports tcl.
further more, why is yosys itself basically scripting of ABC and nextpnr and etc using cpp instead of again using just some reasonable scripting language to connect the pieces?
1. Because you will piss off would-be users. That simple. I have also complained about this endlessly. I also despise Tcl and had to use it to help automate lots of EDA tools in ASIC and FPGA designs, so I'm with you on this. But it isn't a hard question to answer in reality. Users will just be annoyed and think you are annoying and contrarian. (Nextpnr at least allows scripting with Python, and I'd ideally like to see other takes on synthesis in the next few years, thanks to more robust language frontends coming along.)
1.1. It is not "just scripting ABC", but ignoring that. I assume it's because Claire just liked C++. She was historically an embedded/hardware engineer. A lot of them like C++. But I can't read anyone's mind. The project is like, probably 15 years old now (the initial Git checkins from 2013 aren't even full replications of the real history, to my knowledge, which go further back into an ancient SVN server.) What would you have suggested using in 2010? Python with zero static typing and pip?
2. Because actually owning the code you write and ship to your users is a nice and good thing. Dreamplace is a DL-reinforced placer. It's an academic project. Having rode that bull (shipping academic code to production users and then having to rework it again and again), realistically, I'd probably reimplement Dreamplace directly into the codebase if I wanted to ship features to users. No offense but this isn't a SaaS service where the sausage machine is hidden; and most RTL engineers, nor I, want to install some giant annoying shitware ML stack into a python virtualenv to run Nextpnr. Just to be clear, if yosys uses some magical ML reinforcement learning, or doesn't, I don't care. It would be great if it did and it got a bajillion % higher QoR. But I don't want actually using the thing to be a giant pain in the ass.
Most open source software engineering isn't about being Super Epic. It's about meeting users where they are at and handling basic engineering constraints around the project. EDA is no different.
I'm not downplaying the fact that yosys is a logic synthesizer - it's valuable and it's the only one of its kind. But that doesn't change the fact that many of the steps are just piping ABC or nextpnr or whatever and doing it using cpp. See https://github.com/search?q=repo%3AYosysHQ%2Fyosys%20ScriptP.... And like I know this not tangentially but because I tried use yosys but not all of yosys and it was impossible to decompose it.
> No offense but this isn't a SaaS service where the sausage machine is hidden; and most RTL engineers, nor I, want to install some giant annoying shitware ML stack into a python virtualenv to run Nextpnr. Just to be clear, if yosys uses some magical ML reinforcement learning, or doesn't, I don't care. It would be great if it did and it got a bajillion % higher QoR. But I don't want actually using the thing to be a giant pain in the ass.
I'm with you on every instance of this trueism except here, where runtime of the tool is not just important it's absolutely critical for DSE.
> Most open source software engineering isn't about being Super Epic
Again I agree with you in every other instance except here - just like LLVM and gcc can't be just oss, they need to be nuclear weapons grade software in order to have won mindshare (and investment) away from commercial compilers so does yosys.
So you can take me to task for being naive and arrogant but the reality is that the decisions made by yosys prevent further serious investment by anyone that's not playing with toys.
> let me take this opportunity to ask a question about something that makes absolutely zero sense to me: why did yosys insist on sticking with tcl over some more modern/well-known/familiar scripting language? yes everyone has their favorite tcl scripts for vivado or intel or whatever but none of them are transferrable to yosys irrespective of that fact that yosys supports tcl.
First: PYosys exists as Python bindings for Yosys, so you can in fact use a more modern scripting language for this.
Second: even if the Tcl scripts are not transferrable, a lot of our users do still have Tcl experience, and that transfers if nothing else.
But personally? For a lot of tasks you don't need a scripting language. If you want to turn a set of Verilog files into a JSON to pass to nextpnr-ecp5, that's `yosys -p "synth_ecp5 -abc9 -top top -json design.json" *.v` and there is neither Tcl nor Python involved here.
> further more, why is yosys itself basically scripting of ABC and nextpnr and etc using cpp instead of again using just some reasonable scripting language to connect the pieces?
Yosys is not simply some scripting around ABC: you need to parse input files, store this in an intermediate representation (RTLIL), elaborate and monomorphise the input structure, perform optimisation and logic minimisation on it; you need to map large structures like memories, hard multipliers and carry chains (ABC can't do that), legalise flop types and only then can you pass the logic to ABC. Once you get that logic from ABC, you need to turn it into cells that tooling recognises.
That the "script passes" we have that most front-end users use are so simple is because the complex logic is packed into neat passes to call.
We're aware of DREAMPlace, however: DREAMPlace itself is for ASIC applications, so you'd want to use DREAMPlaceFPGA instead, which is...limited to a simplified Xilinx architecture or whatever partial FPGA Interchange Format support they have. We consider speed a feature, and having to deserialise the target device and netlist into FPGA Interchange Format in the hopes that DREAMPlaceFPGA can understand it is...not fast.
So instead we're working on our own multi-electrostatic placer which can more tightly integrate with nextpnr than something using DREAMPlace[FPGA] could.
Reading your answers, I've noticed that there appear to be some major misconceptions about Yosys and how it works: certainly, ABC is a core part of a standard synthesis flow, but Yosys is based around a series of passes that modify a common intermediate representation. That should mean if anything it is very easy to use only part of Yosys.
Obviously it'd be off-topic to try to discuss your particular needs and use-cases of Yosys here, so feel free to email me at lofty@yosyshq.com.
> Reading your answers, I've noticed that there appear to be some major misconceptions about Yosys and how it works
Indeed I opted to go with vivado instead of yosys (so i'm not an expert) but you're being a little disingenuous; there are absolutely places where yosys is explicitly, manifestly, being used as a scripting harness - you literally have ScriptPass
...is your bugbear here that these particular scripts are written in C++ instead of Python?
Yes, those flows use ScriptPass, because for the most part they are scripts. If they were written in Tcl, then Yosys would need to depend on a Tcl interpreter, and equivalently a Python interpreter for Python. By just using C++ for these scripts which usually only developers need to modify, Yosys does not need to depend on either.
But anyway, if I was going to mess about with the flow that much, I'd start by running `help synth_xilinx`, which outputs a list of all the commands that it calls (more or less; it can't model control flow like a proper language can), and then that can be edited into what I need it to do.
> ...is your bugbear here that these particular scripts are written in C++ instead of Python?
this isn't a bugbear - how should i, a user of yosys, that disagrees with the yosys devs about how the xilinx flow should work, proceed?
> By just using C++ for these scripts which usually only developers need to modify, Yosys does not need to depend on either.
you really don't understand how sequence of parameterized steps and compiled binary don't go together at all? see my first question.
> (more or less; it can't model control flow like a proper language can), and then that can be edited into what I need it to do.
again you're being disingenuous - the help message is right there in that file plain as day for everyone to see and the actual steps in the flows are also plain as day and there is an enormous gap in what can be adjusted and exists.
i feel like i'm taking crazy pills here - there's a reason why scripting interfaces are exposed at all right? i.e., tcl. because users need to be able to disagree/decompose/parameterize etc. and these example flows in yosys are frozen in stone (binary) instead of written as scripts that can be adjusted by users.
> this isn't a bugbear - how should i, a user of yosys, that disagrees with the yosys devs about how the xilinx flow should work, proceed?
There is a scripting engine in yosys called yosys script. It doesn't support logic but if you just want to run some commands it works fine.
There is also some basic TCL script support in yosys but I have never used it. So I can't really comment how well it works.
In addition a pass doesn't have to be inside the yosys tree for it to work. You can load any module at runtime. Sure this is not as quick to do then having a full scripting interface but you can be up and running within a few minutes.
I think there is a big difference between published academic tools that work on a specific benchmark format versus an academic tool that has been integrated into a robust EDA toolflow. Also, note that dreamplace is simply accelerated electrostatic placement tool using deep learning frameworks to take advantage of GPUs. Electrostatic placement is not a new concept and is not specific to dreamplace.
> Also, note that dreamplace is simply accelerated electrostatic placement tool using deep learning frameworks to take advantage of GPUs. Electrostatic placement is not a new concept and is not specific to dreamplace.
I don't know why you're explaining this to me (I've made contributions to dreamplace). my point is why reinvent a very well-functioning wheel?
> yes everyone has their favorite tcl scripts for vivado or intel or whatever but none of them are transferrable to yosys irrespective of that fact that yosys supports tcl.
But still it is much easier to have one Tcl script with conditionals targeting all the tools than a myriad scripts in various language-of-the-week. Tcl is just good enough, and extremely "well known".
Yes it is. Some time ago this discussion would have been about Perl, some more time ago it would have been about C shell. If you jumped on all thee languages you'd be in a much terrible situation than you are now. And we are literally talking about EDA here, where you can find way more engineers who know C shell than engineers who know Sh or Bash (or Python).
Well, at least part of this comes down to the fact that we did not choose a scripting language today, but at least a decade ago where Tcl was dominant in EDA and Python would not have been taken seriously.
Awesome, thank you for the details and looking froward to the work with more funding! I have yet to look into the details of nextpnr and this is a great set of of resources for me to get started. I am working on some "deep learning for FPGA implementation" projects and it would be neat to see if I can plug nextpnr into my work. I think (from my point of view), VTR is more of a generic architecture exploration tool / academic tool with a more top down approach hence the arch def files and so on.
I asked whitequark about the name; it's named Glasgow because it's where the headquarters of FTDI are based, and the initial prototypes were intended to be an open-source serial adapter gateware before feature creep set in.
In some sense, yes, using GCC puts limits on what you can legally do. The intent of those limits is well explained by Stallman. In case of API I highly suspect the intent is there. Now, how that intent is interpreted by reading GPL is another matter - though it would be sad to find a company abusing the legal system.
Additionally, I'd like to clarify that nMigen generates Verilog indirectly; it actually generates RTLIL, which is the intermediate representation of Yosys, and then Yosys turns it into Verilog after some cleanup passes.
I'll happily admit to being biased, but nMigen is so much easier for me to work in than Verilog ever was.
Ah, I saw the redirect but figured it was better to use the plain URL. Sadly too late for an edit now.
Hadn't quite gotten the connection with Yosys right yet, as I haven't yet started with nMigen, but yeah I definitely want to go there. But like I said, I prefer getting a good handle on Verilog first so I know what to look for when things go wrong.
I was inspired to all of this by the YouTube video series by Robert Baruch[1], where he tries to recreate the 6800 CPU on a FPGA using nMigen with formal verification.
There are three killers here: self-modifying code, synchronisation, and parallelism, all of which are major headaches for a JIT.
The PS3 does not have self-modifying PPC code (SCEI forbade it), which means the PPE blob can be compiled ahead-of-time (RPCS3 converts it into LLVM). The SPE data can self-modify, however, but (to my knowledge) does not require extensive synchronisation, therefore each SPE core can be put on a thread.
The PS2 code has fairly extensive use of self-modifying code; Naughty Dog in particular will frequently load parts of the executable in and out of memory on both the PS2 main processor (the EE) and the PS1 processor (the IOP), and rely on the synchronisation between these two separate processors to be fairly tight. Trying to make the EE and IOP separate threads running simultaneously breaks this synchronisation, so the EE and IOP have to run on the same thread.
Additionally, the PS2 has two vector units; VU0 is associated with the EE (it can be used as a floating-point SIMD unit in the EE instruction stream) and VU1 is associated with the GS [the PS2's GPU, the Graphics Synthesizer] Interface (GIF) (it can directly output primitives to the GS). This means that VU0 needs to run on the same thread that the EE runs on (because there is instruction stream interlocking), and VU1 needs relatively tight synchronisation to the GS (it is feasible to put it on its own thread, but games can be quite picky with timings)
Sony did manage to ship a PS2 emulator that ran on the second-generation PS3 though (1st generation PS3 had actual PS2 hardware inside, but the second generation was software-only emulation if I recall correctly?). Besides knowing exactly about every hardware detail, any idea how they pulled that off on hardware that was much weaker relative to a PS2, compared to a modern PC?
> There are no plucky garage shops or "wise crowds" making FPGA tools that are even in the same galaxy as ISE or Vivado.
Symbiotic EDA comes to mind. However, I would argue they don't need to be great, just good.
I don't have a copy of Vivado to hand (I have an Altera dev board instead), but I can take PicoSoC with Yosys/nextpnr and have a bitstream in about a minute. If I try the same task in Quartus, that takes about five minutes. Producing a (relatively) inefficient bitstream actually speeds up the compile/run cycle in that scenario.
If the design meets timing, which is often decided by external factors, any extra time spend trying to make the design "better" is wasted time.
But modding a PS2 only requires a memory card with FreeMcBoot on it (it's literally plug and play), and at that point you might as well use an updated version of Linux like frno7's.
This is true. I am mostly pointing out PS2 Linux as an interesting footnote, though it is probably a good argument that PS2 + FreeMcBoot is a better option than modchipped PS1 for MIPS computing.
(actually nextpnr has slowly-maturing support for Lattice MachXO{2,3}, Intel Cyclone V and Gowin parts too)