Slow build times in GHC 8.0.x

[flbulgarelli]

Hi!

I am trying add language-python 0.5.4 to an existing project (mumuki/mulang#161), but it is introducing a terrible time penalty in build times. I have also tried to build language-python HEAD from source, with similar results.

It looks like it is not - at least, entirely - caused by language-python, but by GHC >= 8.0 && < 8.4.1 :

• GHC takes a very long time to compile parser generated by Happy 1.19.8 compared to Happy 1.19.5 haskell/happy#109
• https://ghc.haskell.org/trac/ghc/ticket/14683 - it looks like you have being struggling with this issue, too

I am doing some benchmarks applying antalsz/hs-to-coq@c538e6a to understand if this is also related to https://ghc.haskell.org/trac/ghc/ticket/14766 as pointed out in haskell/happy#109 (comment).

I will keep you posted when I have better, stronger evidence of the issue origin, but on the meanwhile, have you tried any workarounds? Thanks!

[flbulgarelli]

Update: by just building Version2.Parser.Parser at HEAD using stack lts-10.2 (ghc 8.2.22) and adding...

{
...
{-# LANGUAGE CPP #-}

#undef __GLASGOW_HASKELL__
#define __GLASGOW_HASKELL__ 709

module Language.Python.Version2.Parser.Parser (parseFileInput, parseSingleInput, parseEval) where
...

...I managed to build it in 6 minutes. Without that hack, Version2.Parser.Parser has not yet finished - 25 minutes and counting.

[felixonmars]

Just to note that the build time is no longer an issue for me since GHC 8.6.1.

[bjpop]

Thanks @felixonmars. I will try it out.

[bjpop]

I tried compiling with ghc 8.6.1 on OS X High Sierra 10.13.6, cabal-install 2.2.0.0, alex 3.2.4 and happy 1.19.9.

I used cabal new-build to build the code.

Unfortunately the compile time and memory usage was still as bad as before. I don't have exact figures, but it took hours to build and memory usage was in the GBs. So it seems I cannot reproduce your success @felixonmars

If possible can you please report versions numbers, OS, and build commands used.

[felixonmars]

It's on Arch Linux x86_64, ghc 8.6.1, alex 3.2.4, happy 1.19.9. The command is:

    runhaskell Setup configure -O --enable-shared --enable-executable-dynamic --disable-library-vanilla \
        --prefix=/usr --docdir="/usr/share/doc/${pkgname}" \
        --dynlibdir=/usr/lib --libsubdir=\$compiler/site-local/\$pkgid \
            -fsplit-base

[watashi]

One more data point that this is still a problem in ghc-8.6.2.

$ uname -a
Linux localhost 4.18.8-arch1-1-ARCH #1 SMP PREEMPT Sat Sep 15 20:34:48 UTC 2018 x86_64 GNU/Linux
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.6.2
$ alex --version
Alex version 3.2.4, (c) 2003 Chris Dornan and Simon Marlow
$ happy --version
Happy Version 1.19.9 Copyright (c) 1993-1996 Andy Gill, Simon Marlow (c) 1997-2005 Simon Marlow
$ cabal --version
cabal-install version 2.4.0.0
compiled using version 2.4.0.1 of the Cabal library
$ pacman -Qo $(which cabal)
/usr/bin/cabal is owned by cabal-static 2.4.0.0-1

cabal build finished after >48 hours.

[felixonmars]

This is weird, I am able to compile language-python 0.5.4 (with merged GHC build patches for 8.2/8.4) within 1 minute. It used to cost at least 8 hours in comparison.

[LucianU]

I'm trying to build from master inside the Docker image provided by FPComplete and I'm hitting the issue with running out of memory.

@felixonmars, in which environment did you build exactly? Or can you tell me the exact steps you took to get a successful build?

[LucianU]

I finally managed to build this in a reasonable time: ~10 minutes.

I ran everything inside a Ubuntu 18.04.1 LTS VM.
I also added the lines suggested by @flbulgarelli in #41 (comment) to both Language.Python.Version2.Parser.Parser and Language.Python.Version3.Parser.Parser.

[felixonmars]

@LucianU I am building for Arch Linux package, and this is the build script (pure bash): https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/haskell-language-python

I believe I didn't add the above hack.

[LucianU]

Thanks, but as I already said above, I managed to build it in Ubuntu. Without the CPP thing, it didn't look like it would finish soon.

[julian-berbel]

Hi there! I'm getting the same described behaviour. Build is seemingly endless in GHC 8, and adding the aforementioned hack to files Language.Python.Version2.Parser.Parser and Language.Python.Version3.Parser.Parser takes it down to around 5-10 minutes.

I've opened a fork adding these changes. I know it's not ideal, but since this is really just a GHC bug and there's nothing else to do but wait for it to be fixed, maybe you'll consider merging it in the meanwhile?