Tea Leaves for 64-bit Delphi, Part 2

It appears your reviewer spoke about Delphi XE3 instead of Delphi XE2.


Your reviewer was gazing too much into the future and went 2 or 3 years ahead of time.

From educated guessing, Embarcadero will ship with 64-bit compiler preview in 30-60 days* (worse case: 80+ days) from now with time-frame to be around April/May 2011 time. This is evident with a big Embarcadero promotion (presumably to get rid of old stock) currently on right now.

How will the Delphi 64-bit preview fare? Your reviewer looks at all the VCL64 evidence and it looks grim. Unless they do good QA testing, it feels like similar experience to Delphi 2 (read: full of bugs) and took 2+ years to stabilize. (hint: Delphi 5/6/7-era)

They would probably change the Integer size:
Integer = +/- 2^63
LongInt = 2^64;
UInt64 = 2^63;

Correction:
There is no change in Integer size, just pointer size -->
64 bit compiling will not change for certain integer sizes, but only Win32 APIs that integer map pointers. Pointers will be 64 bit.

Correction #2:
There is no i386 FPU levels, masks, op-codes in 64-bit mode.

Piecing the 64-bit jigsaw
Dcc32.exe does a "one-pass compile". That means, many of the symbols are ready-made and resolve at link-time (see: rlink32.dll) immediately.

Dcc32 consists of two parts: dcc120.dll (called from the IDE) and dcc32.exe (called from command-line). The DCU files are intermediate machine language (usually i386 with few optimizations) with preamble (like basic checks, i.e., before a maths operation is called, it calls the overflow check, those that you find on the compiler tab) and restores the CS:EIP (Counter Stack, Instruction Pointer), Data Segment and Stack.

For dcc64, looking at the current integration (assuming they are going with a 32-bit IDE with 64-bit debugger bridge), then they would need to write a debugger bridge that translates __debugbreak() (previously known as Int3) into 32-bit equivalents with to and from 64-bits.

Assuming they did write a VCL/64, they (those at Embarcadero) writing the 64-bit VCL, they would need to roll-back to older versions of the same code (such as IntToStr before it became all-assembly) and translate that to 64-bit equivalents. They could do it both ways: For example, write to SSE2 registers or use NEON (Android/IPhone) registers.

How your reviewer would do 64-bits.If your reviewer was doing a compiler, your reviewer would use the Edison Design Group front-end compiler with back-end Dinkumware for the RTL x64, instead of hacking and slashing through MMX or older assembly optimizations.

Troublesome Console Application?Your reviewer looked at the tiny or very little information about VCL64, DCC64, RLINK64, DCC1xxx64.DLL and other files and wondered what's taking Embarcadero so long?

The trouble seems to be the bridge between 32-bits and 64-bits. In Visual Studio, they use a smaller version of the Visual Studio remote debugger to host the 64-bit EXE and Visual Studio "talks" to the remote debugger interface.

For Embarcadero, either they missed the boat or didn't bother to update TDEBUG32 to TDEBUG64 equivalent for very long time.

For example, if you write "array of [1..65536] of widestring" in your code in Delphi 2005, 2006, this would cause a DCC failure when "Include Remote Debugging Symbols" is turned on -- because array of widestring is not clearly defined in TDBUG32 information.

Who uses TDEBUG32 information? The former TurboPower SysTools, the former MemProof on Automated QA website before it was removed, AQTime, AQ(whatever) products.

Before you can write a debugger, you need to have properly working symbol generation working. Suppose you have to build a remote debugger bridge to bridge between 32/64-bit EXE debugging.

Hence, your reviewer guesses:

1) Delphi 64-bit compiler is missing the debugger when it comes out... or if Embarcadero releases it, it would be half baked or have so many bugs and issues, it becomes impossible to debug a large app.

2) Delphi XE was released on 30th August 2010. Assuming that Delphi XE2 would be released usually 365 + 30 days later (to annoy Delphi developers to update their SA subscription)

3) Delphi x64 compiler preview would disappoint many people. They think it's complete product ready to recompile their favorite app into x64 but it's not -- the next group of people who need to get rid of all the old 32-bit assembly and check their product to work with x64 are the component vendors. The larger vendors (DevExpress) do not even have any Delphi x64 news.

That means, give or take a year to make Delphi x64 stable, it could drag on to 2012 or 2013 more to get a stable compiler... by that time, most of the Delphi developers would have been out-foxed, beaten by their 64-bit competitors, without house, without any way to make more money... making 32-bit EXEs is no longer a marketable asset.

The people who still use Delphi will have to make hard choices when the 64-bit preview comes out...

The less information Embarcadero gives on the compiler development status (other than just being very silent) the more people will start to wonder WTF is happening?

:)

Article notes:
- For non-English speakers: "Tea Leaves" means predicting the future based on known available information.
- Spelling mistakes corrected.
- Pointer information corrected.

Tea Leaves for 64-bit Delphi

Your reviewer was looking at the KSDev purchase and thinking...


Pesky 32-bit assembly and legacy issues.The whole VCL is peppered with 32-bit legacy code, some going even back to 16-bits, such as aliases for WinProcs, WinTypes.


Judging from the last version of KSDEV library before Eugene pulled it from his website, the latest XE codes with some unusual 64-bit codes, some parts of the Embarcadero wiki before they removed it from public view.


Here is your reviewer's take about the current state of Delphi/64.Delphi 64-bits requires a full compiler rewrite. That means, only the language stays the same, everything else doesn't matter.


For example, if the definition of "program", "for", "if", "while", "being", "end" stays the same or similar, it is considered Delphi or Pascal language.


In a liberal sense, that is how RemObjects does it: With RemObjects Prism, the NET library is the main library instead of VCL, there are compatible constructs to make porting code from Delphi/Win32 to NET/Prism.


In order to code for Delphi 64, the whole entire VCL needs to be re-written to first remove the assembly parts and make it as "compiler magic" or in-line assembly routines that is linked to the final 64-bit EXE file.


The first part is easy. Making equivalents to the original Delphi/Pascal codes: Go back a few versions where the IntToStr, and other routines were not converted assembly and use the original codes.


The second part is not-so-easy: The Delphi VCL is limited to Win32 only and nothing for Mac or Linux.


The trick is to make the final result language compatible. Remember the QT/Kylix library, where the Delphi 6/7 had an alternate implementation called QT(whatever).


QT and WxWidget are both respectable libraries, except the fact that the Borland C++ compiler is so far "out of the C++ loop" that it could never hope to have the same respect Microsoft Visual 2008, 2010, Intel C++, GCC and Digital Mars C++ compiler have today.


C++ Builder XE
Talking about C++ Builder is like talking about an athlete who claims it is the "best", won many former awards, with lots of boasting, pomp and grandeur - because of the corporate sponsor decided to stop the athlete from training, making it fat and lazy. The corporate sponsors then decided to silenced the critics by threatening NDA or humiliating them publicly.


In the past 10 or so years, there were certain critics who went to presentations with floating point performance reviews but was stopped last minute because of NDA or Borland threatened lawsuits at them. Some of the critics were so vocal and so disappointed about it, many of them are University professors -- they went and subsequently switch from Delphi to Visual Studio for their teaching syllabus and never looked back.


(To rub insult to injury, C++ builder is almost 90% slower than Visual C++ or even Intel C++ compiler. You can find out easily by running SpecInt, LinPack or the recently compatible Crypto Library to find your own conclusions.)


In relation to QT and WxWidget, there is no or very little documentation about QT for C++ Builder 2010/XE or even WxWidget for C++ Builder 2010/XE.


After heckling with trying to get WxWidget or QT to compile, developers face Interesting problems with C++ Builder: Intellisense sucks, very long compile times, even longer than Visual Studio or Intel C++ compiler, inability to compile code that compiles with Visual Studio/GCC/ICC.


What your reviewer thinks about Delphi XE2 (Delphi Exorcist Edition II)Delphi 32-bit version would have a new VCL library called the UCL library along with the current VCL side-by-side.


In order to make a 64-bit EXE file, the person would have to use UCL exclusively.


Remember how Kylix did it? They had pascal files with QT(whatever) and since Borland could not control the language or state of QT includes, it lead to three completely unusable versions of Kylix.


Since UCL would be cross-platform, it meant instead of TButton, the person would have to re-code their application to use UForm, UButton, URadioButton, UGrid, then to stop the users from doing stupid things, like screwing up the 64-bit registers or doing stupid 32-bit tricks (such as modifying the ES:DI) or the usual assembly code that peppers a VCL application.


The no-assembly issue would make sense. For instance, OSX 64-bit calling conventions or memory management is completely different from Win32 memory management.


Kryukov Software DevelopmentThis would make Mr. Eugene Kryukov, founder of Kryukov Software Development (or KSDEV) very happy person. For many years, the KSDEV VCL libraries does not have a good report writer, good grid and good edit controls. You can be sure DevExpress, Nevrona, Report Builder, LMD, TMS would then make superb components (each sold separately, like toys) built on-top of Eugene's components.


Since the most popular cross-platform libraries would not be used (because of C++ builder issues, GPL licensing issues), Embarcadero decided to use KSDEV because it Mr. Eugene Kryukov was the last person who knew how to write a cross-platform component library and cheap enough for Embarcadero to buy over.


DevExpress is estimated to be worth at least US$20 million, almost the same price as Embarcadero Delphi purchase, making it too expensive for Embarcadero to buy over.


How did Borland value Delphi at almost 70% less than the original US$100 million it sought, when it sold to Embarcadero? The answer itself lies in the way how Borland treated it customers and those on the newsgroup alike: Very shabbily, ignoring complaints and enforcing a SA-damn subscription or annual Delphi updates and payments.

Delphi's Last StandEmbarcadero promises 1st half of 2011 to deliver a working 64-bit compiler. Will they deliver? If the momentum to port Delphi codes to NET, C#, C++ and leave Delphi continues, and they still don't have a working usable 64-bit compiler and library anytime soon, it will be game-over.


It will be game-over because the new Delphi/64 will come with a new UCL component library and everything needs to be rewritten for it to be compatible with Delphi/64. That means -- other component vendors, if they are not in good graces or fell out with Embarcadero (like leaking out Delphi 2010, Delphi 2009 beta builds or uploading Delphi beta builds to chinese FTP sites) will be in for a big surprise.


Those hardest hit will be the component vendors because they need to rewrite their codes to be UCL compatible, the report writer guys need to update the libraries to 64-bit compatible, the grid guy needs to update the grid, the database guy needs to update their database-code and so on...


Those programs which contain custom components needs to be upgraded too, don't forget. Those old libraries which are still used, but the vendor went out of business, they will not be compatible and need significant porting effort to make them work with 64-bits (like making TurboPower Abbrevia, TurboPower Opherus UCL compatible).


Unless you patched KSDEV code before, or posted heavily on KSDEV forum formerly about issues and advised on work-arounds, or wrote Delphi components..., it is going to be tough work doing the code migration...


Most of the Delphi developers don't reach this level because they are so lazy and just buy one component pack after another to solve their problems...


Then, there are the usual gang of idiots who uses warez and pirated cracks, hacks and Delphi-Lovers who use all pirated crap to avoid paying for their libraries. Those people will really feel "Delphi-Love", when, the company went out business, they used a cracked or hacked version of their VCL products and will soon start begging for a non-existent 64-bit version of the same VCL codes. This will become reality soon.

Please see article Tea Leaves for 64-bit Delphi Part 2

:)

64-bit Hopes and dreams fade

The longer Embarcadero waits to deliver 64-bit Delphi, the deeper dreams and hopes fade away...

- Software vendors who write drivers get put-off by Delphi because there is no inter-op between native 64-bit and 32-bit codes. Someone alerted me there was some bridge code to call 64-bit DLLs but not 64-bit COM or OLE.

- 32-bit registry cleaners cannot work correctly in 64-bit Windows 2008R2 or even Vista/64 because they "assume" many things, causing customer's computer to crash afterwards (i.e., you pay for the software and then trashes your computer).

Two Registry cleaner vendors confided with your reviewer how they had no money coming in for past two years and your reviewer urged them to use Visual Studio and rewrite their app.

- Apps that need more than 4GB, like silly newsgroup readers that cannot download more than 4GB of newsgroup data, or Sound/Video software that cannot edit more than 4GB without needing some kind of unnecessary file operations.

- IIS filters, IIS control panel replacements and the same, Apache MODs are dead for Delphi and C++ Builder developers. Even Remy Lebeau (TeamB) was given Visual Studio to replace C++ Builder.

- IntraWeb for IIS/64? Sure, there's a work-around by enabling 32-bit mode but..., requires a dedicated server for this. Even Torry's is not hosted on a dedicated server.


Go to TeamB's website and you will see Craig Stunz's shunning Delphi and using Visual Studio. The whole TeamB blog is full of Visual Studio posts. Go figure.

- 64-bit COM. This is pure sadness. Your reviewer heard that many Delphi developer's add-ins for Microsoft Office no longer works with Office 2010/64 and getting run-over by NET or C++ Add-ins written for Office 2010/64.

Your reviewer even got boiled alive when your reviewer's Office Add-ins could not work and customers were going to drop the product and use some competitor software instead.

How did your reviewer resolve this? No choice, had to rewrite the whole thing in NET wondering when Delphi/64 would be released (2009 passed, 2010 passed, now is 2011 first quarter). In two week's time, many companies will report 2010 Q1 results.

Curiously, there's a company selling Add-ins integration code that links to this blog with notice: Delphi doesn't have an x64 compiler :-(

As hopes and dreams fade, the money also flows away...
Several Delphi developers your reviewer discussed with were devastated when their revenue stream ran out and they had to make painful cuts, like laying off their friends or starting to learn the basket language C# or VB.NET to make ends meet.

Prism solution?
Many Delphi developers who use Prism find employers are not willing to pay another US$499 (or US$560 -- according to several developers in Australia) to buy Prism.

It's either US$499 budgeted to buy an ASP.NET package (NetAdvantage or DevExpress.NET or Telerik) or ... Prism.

Since NetAdvantage does not support Prism (or many forum questions gets unanswered) or any official support from DevExpress, or even many ASP.NET or WinForms component vendors, the answer is very obvious.

Delphi forever... Viva Delphi!
"... I've being a loyal Delphi advocate, i believed on them to deliver on their promises, i went against the trends to keep it alive on the companies I've directed....

But, after seeing Delphi XE, which I will never have the morale to call a new version, or even sell it as such and taking their justifications to be a viable upgrade as a direct insult to me as a customer, I gave up and I am now pointing to the other side of the river.

I'm totally disappointed. I feel sad to see how they are trying to rip off the EU and other countries(Brazil) which are basically keeping the community alive, and how they expect us to even feel comfortable with a line saying that "on my trips to lalala land, they loved the release and they say is the best ever", they know better and its frustrating."
Source

Core Values: 64-bit "Love"

Your reviewer was looking at yet-another-64-bit-thread on Embarcadero's newsgroups and found one particular vendor could not scale beyond 4 gigs.

NewsLeecher, written by Mr. Simon Horup has a serious problem none of his competitors face, his product cannot scale beyond 4 gigs as his product uses all 4 gigs of memory. Mr Simon Horup says:

"I'm having similar problems with the news reader ( NewsLeecher ) I'm developing, using Delphi.
Furthermore, a 64 bit release of our news reader, is one of the most  requested features we get. Literally every single day, users ask us why we haven't made a 64 bit release available yet, referring to our main binary
news reader competitors, who have all released 64 bit versions year(s) ago. And all I can tell them, is that I have no idea whatsoever, when I will be able to provide them what they want in that regards. Something I find really frustating and unprofessional."

Just imagine how you would be in Mr. Simon Horup situation. Everyday, literally someone is asking for a 64-bit version of their product and he cannot deliver such product. That means lost sales, lost time and need to port to either MFC or C# or Java... (either way, at least C# and Java can scale)

In the weeks, months to come... Mr. Simon Horup will probably lose all his customers... because his product cannot utilize the millions of posts on those usenet forums.

Everyone is starting to ask for a 64-bit executables. If Embarcadero cannot deliver an x64 compiler soon... there will be no money coming in to pay for the bills, the wages/salary for the developers, might as well get another job or learn another language...

VB.NET, C#, WinForms, WebForms, MFC C++, QT C++ compiles in less than one second, works with 64-bits and is here today...

:)

Tea Leaves - Feels like Delphi 2011 coming within 60 days

Your reviewer got several emails from Embarcadero.

Every year, without fail, Borland, then DevCo, then CodeGear, now Embarcadero would send these ridiculous discount offers to buy "older" versions of Delphi. You might be thinking, that's out of your mind, but wait...

The proof comes from 5 years of marketing data:
- In Year 2005, they offered discount pricing on Delphi 2005 on October period with "hints" towards buying SA. Delphi 2006 was released on November 23rd, with many people fuming mad they brought older versions of Delphi.

- In Year 2007, they offered discount pricing on Delphi 2006 on February period with "hints" towards buying SA. Delphi 2007 was released on 16th May, with many people fuming mad they brought older versions of Delphi.

- In Year 2009, they offered discount pricing on Delphi 2009 on June-July period with "hints" towards buying SA. Delphi 2010 was released on 15th August, with many people fuming mad they brought older versions of Delphi.

The year is 2010 with Embarcadero giving ridiculous prices - Buy Delphi 2010 and get Full RAD studio, TMS controls, and Delphi 2010 handbook. (How about with free license to Delphi 2011 as well?)

Around 2 months from now, your reviewer won't be surprised Delphi 2011 will be released, and most people who brought Delphi 2010 without SA will be fuming mad they licensed "older versions" of Delphi instead.

Why two months period? So people who brought Delphi 2010 won't be entitled to free version of Delphi 2011.

Don't be surprised there will be Nick Hodges telling them to upgrade or buy SA to protect their investment, same like what happens every year (for past 3 years) around Delphi upgrade time.

And of course - the same vendors will hint of upgrade time - ReportBuilder upgrade time, InfoPower upgrade time to name a few vendors who post upgrades on each Delphi version.

:)

Delphi 2011 - Beta Leaks

Your reviewer was amused over "URL links" to download beta version of Delphi, "Fulcrum". Since there will be a supposed public beta-test, maybe this information is irrelevant.

Here are some suggestions to prevent leaks:
a) When making a beta site download link, remember to put a user-specific login/password link to download. Meaning: You make a password protected site to download the stuff, but do not make password-protected links. Make it GEO-IP protected, so the last IP downloaded are known. For example, someone in USA should not be downloading from China. :)

Some clarification:
- There's a login/password to HTTP web pages download links
- There is no login/password to download the Beta files
There should be login/password to download the Beta files.

b) Make people understand they will lose their BETA rights if they blog about it and even get prosecuted if they leak out links.

[Corrections: Clarification added]

Reading tea leaves - Delphi 2011 or maybe Delphi 2010 update 8/9

Delphi 2011 to include MacOSX, Linux platform, on Windows:
Your reviewer was looking at this post by David Intersimone from Add-in-Express blog. I am surprised David keeps saying "compiler" without "IDE", new "VCL".

Your reviewer found out why. Marco Cantu's Delphi 2010 book, quotes:
"Despite three versions, Kylix never caught on, because of some instability of IDE. That's probably one reason why Embarcadero mentioned its future cross-platform versions of Delphi will use the Windows IDE and let developers cross-compile to other operating systems."

That means, use back same old IDE but generate a MacOSX or Linux ELF file from Windows and (probably) do remote debugging to debug it.

Where's 64-bits?
From the road-map, maybe the next version of Delphi will come out in 2012 and then maybe Delphi/64 in Delphi 2013 or 2014... Of course, your reviewer could be wrong, but playing devil's advocate:

a) There is completely no blog, no article, no post from Embarcardero's website about 64-bit assembly, 64-bit stacks, 64-bit X64 instruction set usage (sneak-peek previews)

b) There are no 64-bit details from those who know, such as Marco Cantu's blog, Winds of Wings, etc.

c) There is no decent documentation for x64-bit handling.

d) Looking at how lonely Rudy Third Dental Probe Third Rail for Ruby newsgroup, Delphi for PHP newsgroup and Delphi for NET newsgroup has been, it probably means poor sales. If this new MacOSX (32-bit only, assuming) and Linux (32-bit-only) bombs or still results in poor sales...

... there may never be a Delphi/64 because there is not enough money to finance it ... and the Delphi people are really sick of paying for subscriptions, upgrades.

64-bit Office is coming to town, Delphi applications going to get killed soon

While Team B is busy picking fights on almost everyone who discuss about 64-bits, be it, 64-bit registry issues, 64-bit explorer issues, need for 64-bit, pros & cons of 64-bits, there's very very big issue coming soon.


Ignorance is bliss
Your reviewer will laugh at many Delphi vendors whose 32-bit version could not compete against their 64-bit foes:

1) Registry Cleaners. Hey bro, did you see any Delphi 64-bit registry cleaners? Many Delphi vendors lost out when they could not "fix" 64-bit registry issues.

2) Video/Sound editors. Did you see that cool 64-bit Video/Sound editor that had no 4gigs limitation, could work twice as fast on 64-bit PCs and outsells every stupid 32bit application on the market? This is reality now.

3) Explorer integration. This is self-explanatory. Even the dogs could lick Lazarus sores and using Free Pascal, you could write for 64-bit DLLs...


64-bit Office coming soon
Almost all Add-ins today are built for 32-bits, and with Delphi unable to build 64-bit COM DLLs, many Delphi developers who used Word-COM-integration, Excel-COM, PowerPoint-COM will find they urgently need to move to 64-bit soon...


Cross-Platform, Linux, Mac vs. 64-bits
Which is more important? Making DCG (Delphi Code Generator) back-end for Code Generator emit cross-platform code or 64-bit issue?

Your reviewer thinks 64-bit Delphi compiler will become so important now, if there is no 64-bit compiler, all those fancy business Add-ins with Office Integration will soon go out of business.


When will Delphi/64 be out? Maybe when you don't have any money left
They promised Delphi/64 - 5 years ago, every year - new rants, more complaints (and more TeamB insults). The real truth, nobody cares because everyone is on subscription, whether or not Embarcardero takes 2 or 3 years more, everyone has to pay SA/ or new license (since they no longer honour old discounts). By the time 64-bit Delphi came, all your code-base either move to C#/64bits (most likely), C++/64bits, Python/64, PHP/64bits or unlikely, RealBasic/64bits.


Your reviewer thinks he's like Charlie Frost in 2012 reporting live, about how everyone changed from 32-bits to 64-bits, and stayed behind to broadcast about it, and got killed in the blast.

Think about it, while everyone is moving towards 64-bits, are Delphi developers left behind?

Delphi 64-bits... in 2011 or maybe 2012 or 2013

Delphi 64-bit issue is gonna hurt lots of people...

The first problem people "notice" are 64-bit issues, such as, Explorer Integration, System Services, IIS 64-bits, 64-bit "aware" utilities.

- Let me start with Explorer integration. other than ShellPlus (you can read about it on this blog), I could not get my delphi-made 32-bit Explorer to work in Windows 64-bits. (let's not talk about Windows CE devices).

(There's FreePascal 64-bits, but if you can get a working decent explorer integration to work correctly without crashing and requiring so many DLLs).

- There's IIS 64-bits. Did anyone recently hear about php 64-bits, perl 64-bits, ASP.NET 64-bits? maybe I'll wait until every one gets rich from their web-applications and my Delphi-based Interweb-application is still "under-development"...

- There's other nifty system utility applications that needs native 64-bit access, such as Registry Editors, Driver front-ends, Database front-ends that simply don't play well in 64-bits mode.

For example, there are those popular registry editors... that don't work well in 64-bits... and getting killed by those C/C++-based 64-bit Registry vendors. There is no easy way to access 64-bit Windows registry low-level from 32-bit modes...

- Another touted development model is the driver-front-end, where the driver guy spends months writing those drivers and then makes a quick front-end in Delphi... With Delphi 64-bits coming maybe in Delphi 2010 or 2011, we'll "drop" Delphi and just use native C++ to write those front-ends. oh well...

- The biggie is 64-bit IIS mode, 64-bit apache, where the ISAPI DLL or Apache module needs to be in 64-bits mode to work... You can run a 32-bit app in 64-bit IIS, but when you want to compete with big-iron C++ vendors, you need full 64-bit mode. Most of the delphi guys write those ISAPI filters and small web-server utilities... I've never seen a 64-bit ISAPI filter written in Delphi or web-server control panel written in Delphi either.

but wait, there's more. suppose you want to make a plug-in to a 64-bit application, such as 64-bit MSI, or 64-bit adobe, wait, maybe i'm getting ahead of my self...

Corrections:
This article came out before Delphi 2010 came out. Delphi 2010 and no 64-bit Delphi, oh well...


Article update 2013 March -
This article now contains invalid information since FreePascal has x64 support and they fixed bugs with their 64-bit integrations. Delphi XE2 with 64-bit was released. If you are using Delphi 2010 or Delphi XE, you should upgrade. Article contained correct information at that time.