Just a quick reminder to everyone publishing Delphi projects with source:
Please don’t publish your .dproj & .groupproj, only publish the .dpr & .dpk
The reason? Those files include machine specific settings, such as paths, DCU/DCP/BPL/EXE output directories, along with your favorite debug & release options, which are likely different from that of your fellow developer.
It’s possible to have them manually cleaned up, but that’s tedious and error-prone short of checking their xml content manually.
Pretty much every single project with a .dproj out there has issues: that’s from major open-source projects to Embarcadero’s own samples. None of them (of you) got all of them cleaned up right.
But even getting the published .dproj right doesn’t matter: .dproj is where compile options are stored, options you’re just bound to change and adjust. When those .dproj are in a project you synchronize with via version control (SVN, GIT, etc.), your locally modified .dproj will likely conflict next time you synchronize, sometimes in unintended and not immediately obvious ways.
Hopefully in a future version, Embarcadero will split the .dproj, so that machine-specific settings are in a distinct file from the non-machine specific settings, which would essentially be per-project relative paths to the source files.
Ad interim, .dproj are just a kludge by design.
I totally disagree.
Not delivering proper(!) dproj and groupproj files is the root of all problems with installing 3rd party components that a big part of the Delphi developers (especially the not so experienced ones) have.
When I look at Delphi forums and newsgroups there are a lot of problems and questions with installing virtual treeview and other famous components just because searchpaths and output directory are not correctly set up. Often this manifests in big problems when you are using different Delphi versions on the same machines.
Even more important with multiple platforms and outputs in XE2. How do you set up the dcu output path for 32-bit and 64-bit without a dproj file? The dcu files end up being compiled next to the pas file by default. This results in recompilation every time you switch the platform in your project or you might end up with strange effects like the famous “Unit1 was compiled with a different version of Unit2” Error.
@Stefan Glienke
Doubtless paths are a common problem, but .dproj/.groupproj are just the wrong way to address it: they just create even more problems down the road.
What’s needed is another, distinct options file, guaranteed with no machine-specific settings, and attached to the source, rather than a project. Or a Java-like directory-based approach.
The reason for the above is that even if you define paths in the .dproj of your package, that’ll do nothing for other projects in which you wish to be able to use & access the source (which is like, pretty much all the time): you’ll still need to setup browse paths.
DCU & others outputs directories should not and never be defined in the the .dproj IME because that just leads to every project following different conventions.
They should be defined across projects, and the only way to do that is currently to define them in your default project options, and not have/use the .dproj delivered, that way when the IDE recreates the .dproj, it takes your default options.
That’s still not perfect, as you can still manually override those in a project, but it greatly reduces the risk of having every project go with its own conventions on destination directories.
@Eric
I will explain how I do it usually. I define the output folders for the package to some Lib\ folder. Usually that folder is somewhere in the folder of that component or library.
Then I add that folder to the Library path in Delphi and the source folder to the browsing path. If you want to be able to step through this libraries sources I also add the output folder with the debug dcu files to the debug dcu path.
This way you only need to compile this library/package/component one time (and of course everytime you update the sources).
When you want to use that component/library in your project you don’t need to do anything, just drop the component and/or add the unit. No library path, no manually adding the required units, nothing. In XE2 you also can switch easily between 32-bit, 64-bit or OSX without having trouble. That of course requires setting up the mentioned paths at the first time. It never recompiles any of the sources of that 3rd party library/component when you use it in some project.
This is why I actually like they changed the default dcu output path to not be next to the pas files because that way you cannot accidently rebuild 3rd party source when not needed (by accidently I meant clicking “build project” not knowing that it recompiles EVERYTHING it can find)
Actually this is not only the way I do it is what most 3rd party components setups do:
– putting binaries in a different folder than the source
– adding the path to the binaries (this also requires adding dfm and some other resource files to that folder)
– adding the source path to the browsing path
– optionally adding a debug version of the binaries and the path to the debug DCU path
@Stefan Glienke
I meant Lib\<DelphiVersion> folder
The solution is to create environment variables that define root paths and never put full paths in your project config – only paths relative to current directory, or relative to a root path environment variable.
@Stefan Glienke
Accidental rebuilding isn’t an issue with a compiler as fast as Delphi, but lack of simple rebuilding capability is an issue, if only to be able to build against the user’s desire (debug? release? stack frames? range checks? assertions? etc.).
@Lars Fosdal
Environment variables are just another bag of trouble, f.i. one XE2 environment variables conflicts with a variable defined by HP on laptops, setting the XE2 value borks some HP upgrade utility, leaving the HP value borks XE2. Also they need to be adjusted manually, don’t go through versionning systems, can be adjusted by other software, etc.
The dotted unit names could have been a great opportunity to solve that, by allowing directory-based name-spaces, but they just made it into a cosmetic hack (and a pointless PITA when maintaining source that has to run in both XE & XE2)
I’m talking about the BDS-internal User Override environment variables, not system wide.
(Tools, Options, Environment variables, User overrides)
In my company were are using subst command from windows e.g. “subst x: ”
So the first one can check-out the repository to c:\development\svn and the other on to c:\mydirectory
This works very good.
@Lars Fosdal
You can still have collision with whatever your build tools use during the build, for instance the particular collision for XE2 is
http://delphihaven.wordpress.com/2011/09/01/xe2-or-more-exactly-msbuild-vs-hp-notebooks/
If every Delphi library out there has to define an environment variable for its paths, even BDS-internal, the risk of collisions goes way up.
Environment variables are a hornet’s nest 🙁
We’re also using subst here, that’s great for internal code (common paths for everyone).
That alas doesn’t give a solution for third party that have to publish to developers “in the wild”.
@Eric
First of all you can easily have a debug and a release version as I mentioned earlier. If you want to specify the compiler settings more detailed you can still add the source directory to the project you are working on and be able to rebuild them depending on what your settings are. But I am very sure most developers don’t need that by default. Even less for 3rd party libraries.
I actually prefer making things as easy as possible for the users of my library. If you are some poweruser that is making specific settings that differ from the default you most likely are able to modify my package or use your own version of it.
@Stefan Glienke
You can have both a debug and release version, but that involves more work in setup, and when using, it means you have to go to another project to adjust the compiler options to your liking…
All in all, I really don’t see why bother, if it was C++ and very slow compiles, sure, but in Delphi? Just not worth the effort.
Also, I don’t think it’s simpler that way, since it means that after each update or synchronization with SVN/GIT/whatever, you have to think and manually go recompile those packages, even for minor fixes that aren’t interface-breaking.
If all the files are in the path, that’s no problem, it’ll happen automagically. And if nothing changed, they won’t be recompiled unless you build, and even then, you’re looking at seconds lost (if that much).
We used to think like this. Then we started using relative paths and put all libraries in Version control.
It does not matter which directory the code is checked out too.
Now we are stability in build settings if a specific project needs something special set in the DProj we all get the changes.
Delphi has a fast compiler compared to C++, but people still want it to compiler faster. If we would configure Delphi to see all source files, our compile times would climb up a lot. Compiling over 20 million lines of code takes its time. 400,000 LOC are compiled much faster. And with the IDE’s “Project dependencies” the IDE decides if it needs to recompile those packages. The developer doesn’t need to remember to recompile all packages after a working copy update.
For D2009 and later you already have the means to accomplish most of what you’re looking for. Create an option set file (*.optset) that contains all your machine specific settings and apply it to the base configuration of your project. The DPROJ can now be left free of machine specific settings and checked into version control while you keep the OPTSET on your machine only.
If another developer starts they check out the code (including the DPROJ) from version control then take a manual copy of your OPTSET file and modify it to suit their machine.
@Andreas Hausladen
At least in the early days of Delphi XE, project dependencies rendered the IDE unstable, has it been fixed since? I’ve tried that on and off since project groups got added, never worked in a stable fashion AFAICT.
In terms of performance, I’m dealing with million-lines-plus projects, regular compiles are a matter of seconds with thousandths of units in the search path, builds a matter of 30 seconds, so I’m more than willing to gain stability and convenience for a couple of seconds per compile, if that much…
@Lachlan Gemmell
OPTSET is a per-project thing, what I’m referring to is a per-source-folder thing, sort of like what you can have in Java, so that if you have 100 projects, you don’t have to deal with 100 optset/dproj which duplicate the same info all over the place.
Think of it like a “namespace” manifest file, which you would place in the root of a project folder and that would list the relevant source subfolders.
It would also open up “uses namespace.*” and similar scenarios, and get us proper namespaces (with added value), rather than the kludgy ersatz of a namespace currently available (where the ‘.’ in unit names could just as well have been a ‘_’ for all we get out of it).
We are using Project Dependencies in Delphi 2009. The only thing you have to be careful with is that if you copy a dproj file, that you change the project’s GUID, otherwise the dependencies won’t work or you get an infinite loop in the IDE.
The problem with “namespace.*” is that Delphi can’t handle cyclic unit references. So you would have to guarantee that all units in a namespace do not reference units in namespaces that use those units. A namespace could be seen as a kind of (runtime) package where the dpk file is auto-generated from the directory listing.
That would make projects a lot more modular than they used to be (especially project that grew over 15 years). The developer would have to think before he puts a new class or function into a namespace. This might be an exercise for my Christmas vacation (loading only a fake project, replacing Delphi’s IDE build process + msbuild script, tricking CodeInsight into working on a “namespace-package”, finding a way to trick the IDE that the fake project contains all files)
Having some compiler-side way to restricts cross-dependencies between libraries/namespaces would be a boon, currently you have to manually do that, and it’s not exactly simple, especially as the compiler tends to complain about missing units or unreferenced package units in places different from where the issue is introduced.
Crazy. Bad. Stupid idea.
If you don’t want to ship DPROJ files with your sources, at least tell people the search paths, and component dependencies, so that the person building your code doesn’t have to do it by hand.
Paths that are hardcoded (C:\dev\comp\folder1) are better than NOTHING. I can fix prefixes easily. I can’t invent something out of nothing.
Delphi code is brutal to get working on a machine other than the one it was built on because Delphi developers in 2011 haven’t even got as far in terms of “build sophistication” as most 1970s era Unix C developers were, with make, autoconf, etc.
Giving up isn’t the answer. Tools to make delphi environments portable, and to make building this easy, are. DPROJ files, plus some judicious use of environment variables, and you’re good enough.
If you want a .DPROJ clean/scan tool to normalize your paths (C:\DEV –> $(DEV)) then fine. But don’t give up.
W
Incidentally, dproj.local is where REALLY single-developer information is meant to be. And I believe that much of it is. However my guess is that you and Embt don’t agree about search paths being “single machine specific”.
W
Not my fault, blame whoever designed the .dproj, or abused the design.
In all libraries out there, those are usually obvious to figure out, and can be added in a few click of the paths dialogs.
No, they’re worse, and your suggestion of replacing them by hand is even worse: first, no, you can’t prefix them “easily”, not more easily than just selecting the folders in the path dialog that’s for sure, and second, it just won’t float if you synchronize the source to a repository (you’ll just get conflict at every synch if you’re a subscriber, and if you’re a committer, you’ll mess with other’s .dproj, both paths and compile options).
Come on! Using an environment variable is NEVER judicious, or justifiable for that matter, even in the last century it wasn’t. It’s a kludge, period.
I don’t doubt it was all designed with some good intentions, but at some point, the design was forgotten, or overlooked, or abused, and maybe not properly documented, so whoever abused it may not be exactly at fault.
Regardless, in the end, you have to be pragmatic: .dproj are best not shared. Nobody seems to get them right, EMBT included.
And the alternative takes only a few seconds to add paths with the dialog the IDE provides. One time. Not something you have to manually fix every time you synchronize with SVN/GIT/whatever.
And that’ll work for different projects, including new projects, while .dproj is intrinsically project-specific, so you should only place project-specific paths in it in the first place.