Started implementing tests

2022-09-25 22:13:05 +02:00
19 changed files with 72 additions and 351 deletions
--- a/README.md
+++ b/README.md
@ -1,84 +1,3 @@
 # libinstaller

-This Python tool has been developed to automate the installation of libraries frequently used by HPC applications.
-
-## Basic Usage
-To install the library `<lib>` and all it's dependencies (if there are any) using compiler `<compiler>` and MPI `<mpi>`, make sure to have a configuration file `<lib>.json` in the `config` directory and invoke the tool as follows:
-```
-./libinstaller --compiler <compiler> --mpi <mpi> --<lib>
-```
-To install all libraries with build recipes in `config`:
-```
-./libinstaller --compiler <compiler> --mpi <mpi> --all
-```
-For further supported options and their default values, please have a look at `./libinstaller -h`:
-```
-usage: libinstaller [-h] [--config CONFIG] [--prefix PREFIX] [--src SRC] [--work WORK] [--keep-work] [--compiler COMPILER] [--mpi MPI] [--threads THREADS] [--verbose] [--separate-lib64] [--disable-shared] [--all]
-
-options:
-  -h, --help           show this help message and exit
-  --config CONFIG      Path to config directory [$pwd/config]
-  --prefix PREFIX      Path where install directory should be generated [$pwd]
-  --src SRC            Path where to download source code to [$pwd/src]
-  --work WORK          Path to working directory for builds [$pwd/work]
-  --keep-work          Disable removal of work directory after successful builds
-  --compiler COMPILER  Select compiler (gnu, intel, aocc) [gnu]
-  --mpi MPI            Select compiler (hpcx, intelmpi, openmpi) [hpcx]
-  --threads THREADS    Number of threads used for make [8]
-  --verbose            Print build output to screen instead piping it to logfile
-  --separate-lib64     Do not create symbolic links of files from lib64 in lib
-  --disable-shared     Disable building of shared libraries
-  --all                Install all libraries with config file in config/
-  --ignore-deps        Do not add dependencies of libraries to the install process
-```
-For each library configuration file available in `config`, the corresponding `--<lib>` and `--<lib>-version` options are added to the parser automatically, eg:
-```
--hdf5                          Enable build of hdf5
--hdf5-version HDF5_VERSION     Set hdf5 version [1.13.2]
-```
-The name of the installation directory is automatically generated and has the form
-```
-<prefix>/inst-<compiler>_<compiler_version>_<mpi>_<mpi_version>
-```
-
-## Library configuration files
-The `config` directory contains various json files describing how to obtain and build the corresponding library. It must have the following structure (all keys are mandantory):
-```
-{
-    "name" : "mylib",
-    "dependencies" : "lib_a,lib_b",
-    "default version" : "x.y.z",
-    "download" : "git clone --depth 1 --branch hdf5-$VERSION https://myurl.com/mylib.git mylib-$VERSION",
-    "configure" : "autoreconf -fi ; ./configure --config-option --prefix=$PREFIX $SHARED",
-    "build" : "make -j $BUILDTHREADS",
-    "install" : "make install",
-    "object files" : "123"
-}
-```
-### name, default version
-On startup, the installer tool will read the configuration files in `config` and add coreesponding options to the argument parser. In case `--<lib>-version` is not set, the default version from the json file is used.
-
-### dependencies
-Dependencies are used to determine the best installation order of the selected libraries to satisfy all dependencies.
-When the installer encounters a library to install, all dependencies are also added to the installation process if not already selected. In case `--ignore-deps` is set, the dependencies need to be activated manually when invoking libinstaller or be present in `LD_LIBRARY_PATH`.
-Note that `$PREFIX/lib` is automatically added to `LD_LIBRARY_PATH`
-
-### download
-Download command. This is being executed in a subshell. Make sure that it works in the shell used to start libinstaller.
-The command may contain the variables `$VERSION` which will be replaced by the selected code version of the form `XXX.YYY.ZZZ` or `$VERSION_UNDERSCORE` which will be replaced by `XXX_YYY_ZZZ`. 
-The files will be downloaded to the `<src>`directory
-
-### configure
-Configure command. This is being executed in a subshell. Make sure that it works in the shell used to start libinstaller. 
-The variable `$PREFIX` will be replaced by the installation directory. The vairable `$SHARED` will be replaced by `--disable-shared` in case the respective option is set when invoking libinstaller.
-
-### build
-Build command. This is being executed in a subshell. Make sure that it works in the shell used to start libinstaller. 
-The variable `$BUILDTHREADS` will be replaced by the number of threads set when invoking libinstaller (`--threads`).
-
-### install
-Build command. This is being executed in a subshell. Make sure that it works in the shell used to start libinstaller. 
-The variable `$PREFIX` will be replaced by the installation directory.
-
-### object files
-The number of `*.o` files generated during compilation. It will be used to draw progress bars in case `--verbose` is not set.
+A simple Python tool to install libraries with commonly used compiler- and MPI combinations 
--- a/config/aec.json
+++ b/config/aec.json
@ -1,10 +0,0 @@
-{
-    "name" : "aec",
-    "dependencies" : "",
-    "default version" : "1.0.6",
-    "download" : "git clone --depth 1 --branch v$VERSION https://gitlab.dkrz.de/k202009/libaec.git  aec-$VERSION",
-    "configure" : "autoreconf -iv; ./configure --prefix=$PREFIX $SHARED",
-    "build" : "make -j $BUILDTHREADS",
-    "install" : "make install",
-    "object files" : "10"
-}
--- a/config/curl.json
+++ b/config/curl.json
@ -1,10 +0,0 @@
-{
-    "name" : "curl",
-    "dependencies" : "",
-    "default version" : "7.85.0",
-    "download" : "git clone --depth 1 --branch curl-$VERSION_UNDERSCORE https://github.com/curl/curl.git curl-$VERSION",
-    "configure" : "autoreconf -fi; ./configure --with-openssl --prefix=$PREFIX",
-    "build" : "make -j $BUILDTHREADS",
-    "install" : "make install",
-    "object files" : "362"
-}
--- a/config/eccodes.json
+++ b/config/eccodes.json
@ -1,10 +0,0 @@
-{
-    "name" : "eccodes",
-    "default version" : "2.27.0",
-    "dependencies" : "zlib,szip,curl,aec,xml2,netcdf-c,netcdf-f,hdf5",
-    "download" : "wget https://confluence.ecmwf.int/download/attachments/45757960/eccodes-$VERSION-Source.tar.gz ; tar zxf eccodes-$VERSION-Source.tar.gz ; mv eccodes-$VERSION-Source eccodes-$VERSION",
-    "configure" : "mkdir build; cd build; cmake .. -DCMAKE_INSTALL_PREFIX=$PREFIX",
-    "build" : "cd build; make -j $BUILDTHREADS",
-    "install" : "cd build; make install",
-    "object files" : "515"
-}
--- a/config/hdf5.json
+++ b/config/hdf5.json
@ -7,4 +7,4 @@
    "build" : "make -j $BUILDTHREADS",
    "install" : "make install",
    "object files" : "1136"
-}
+}
--- a/config/netcdf-c.json
+++ b/config/netcdf-c.json
@ -1,10 +0,0 @@
-{
-    "name" : "netcdf-c",
-    "default version" : "4.7.4",
-    "dependencies" : "zlib,szip,curl,hdf5",
-    "download" : "git clone --depth 1 --branch v$VERSION https://github.com/Unidata/netcdf-c.git netcdf-c-$VERSION",
-    "configure" : "./configure --prefix=$PREFIX CPPFLAGS=-I$PREFIX/include LDFLAGS=-L$PREFIX/lib $SHARED",
-    "build" : "make -j $BUILDTHREADS",
-    "install" : "make install",
-    "object files" : "435"
-}
--- a/config/netcdf-f.json
+++ b/config/netcdf-f.json
@ -1,10 +0,0 @@
-{
-    "name" : "netcdf-f",
-    "default version" : "4.5.3",
-    "dependencies" : "zlib,szip,hdf5,curl,netcdf-c",
-    "download" : "git clone --depth 1 --branch v$VERSION https://github.com/Unidata/netcdf-fortran.git netcdf-f-$VERSION",
-    "configure" : "./configure --prefix=$PREFIX CPPFLAGS=-I$PREFIX/include LDFLAGS=-L$PREFIX/lib  $SHARED",
-    "build" : "make -j $BUILDTHREADS",
-    "install" : "make install",
-    "object files" : "60"
-}
--- a/config/netcdf_c.json
+++ b/config/netcdf_c.json
@ -0,0 +1,4 @@
+{
+    "name" : "netcdf-c",
+    "default version" : "4.2.1"
+}
--- a/config/netcdf_f.json
+++ b/config/netcdf_f.json
@ -0,0 +1,4 @@
+{
+    "name" : "netcdf-f",
+    "default version" : "4.6.0"
+}
--- a/config/xml2.json
+++ b/config/xml2.json
@ -1,10 +0,0 @@
-{
-    "name" : "xml2",
-    "dependencies" : "",
-    "default version" : "2.8.0",
-    "download" : "git clone --depth 1 --branch v$VERSION https://gitlab.gnome.org/GNOME/libxml2.git xml2-$VERSION",
-    "configure" : "./autogen.sh ; ./configure --prefix=$PREFIX $SHARED",
-    "build" : "make -j $BUILDTHREADS",
-    "install" : "make install",
-    "object files" : "133"
-}
--- a/lib/dependency.py
+++ b/lib/dependency.py
@ -1,102 +0,0 @@
-import os
-import glob
-from lib.io import load_lib_data
-
-def topological_sort(source):
-    pending = [(name, set(deps)) for name, deps in source]
-    emitted = []
-    while pending:
-        next_pending = []
-        next_emitted = []
-        for entry in pending:
-            name, deps = entry
-            deps.difference_update(set((name,)), emitted)
-            if deps:
-                next_pending.append(entry)
-            else:
-                yield name
-                emitted.append(name)
-                next_emitted.append(name)
-        if not next_emitted:
-            raise ValueError("cyclic dependancy detected: %s %r" % (name, (next_pending,)))
-        pending = next_pending
-        emitted = next_emitted
-    return emitted
-
-
-def sort_libs_by_dependencies(selected_libs):
-    # no need for sorting in case of only one lib
-    if len(selected_libs) < 2:
-        return selected_libs
-
-    lib_names = []
-    for lib in selected_libs:
-        lib_names.append(lib['name'])
-
-    deplist = []
-    for lib in selected_libs:
-        name = lib['name']
-        # only sort after dependencies that are actually present to avoid cyclic dependencies
-        deps = []
-        for dep in lib['dependencies'].split(','):
-            if dep in lib_names:
-                deps.append(dep)
-        dependencies = set(deps)
-        if dependencies == {''}:
-            dependencies = {}
-        deplist.append([name, dependencies])
-    sorted_deplist = topological_sort(deplist)
-
-    sorted = []
-    for entry in sorted_deplist:
-        for lib in selected_libs:
-            if lib['name'] == entry:
-                sorted.append(lib)
-    return sorted
-
-
-def load_dependencies(library, selected_libs, config_dir):
-    lib_names = []
-    for lib in selected_libs:
-        lib_names.append(lib['name'])
-
-    dependencies = library['dependencies'].split(',')
-    for dep in dependencies:
-        if len(dep) > 1:
-            if dep not in lib_names:
-                config_file = config_dir + "/" + dep + ".json"
-                if os.path.exists(config_file):
-                    dep_data = load_lib_data(config_file)
-                    dep_data['version'] = dep_data['default version']
-                    selected_libs.append(dep_data)
-                else:
-                    print("Warning - library " + lib['name'] + " depends on " + dep + " for which no configration file was found. Presence in LD_LIBRARY_PATH is assumed.")
-
-
-# load library data from names in argparse result
-def load_selected_libs(config_dir, arg_namespace, args, install_all_libs, ignore_deps):
-    selected_libs = []
-    if install_all_libs:
-        for config_file in glob.glob(config_dir+"/*.json"):
-            data = load_lib_data(config_file)
-            data['version'] = data['default version']
-            selected_libs.append(data)
-    else:
-        ignore_names = ["config", "mpi", "compiler", "prefix", "src", "work", "keep_work", "threads", "verbose", "version", "disable_shared", "ignore_deps"]
-        for lib_name in args:
-            if lib_name not in ignore_names and "version" not in lib_name:
-                install = getattr(arg_namespace, lib_name)
-                if install:
-                    version = getattr(arg_namespace, lib_name + "_version")
-                    lib_name = lib_name.replace('_','-')
-                    config_file = config_dir + "/" + lib_name + ".json"
-                    data = load_lib_data(config_file)
-                    data['version'] = version
-                    selected_libs.append(data)
-
-    if not ignore_deps:
-        # also add all dependencies to the install list
-        for lib in selected_libs:
-            load_dependencies(lib, selected_libs, config_dir)
-
-    return selected_libs
--- a/lib/init.py
+++ b/lib/init.py
@ -25,12 +25,10 @@ def init():
    config_parser.add_argument('--compiler', help='Select compiler (gnu, intel, aocc) [gnu]', default='gnu')
    config_parser.add_argument('--mpi', help='Select compiler (hpcx, intelmpi, openmpi) [hpcx]', default='hpcx')
    config_parser.add_argument('--threads', help='Number of threads used for make [8]', default='8')
-    config_parser.add_argument('--verbose', help='Print build output to screen instead piping it to logfile', action='store_true')
+    config_parser.add_argument('--verbose', help='Print build output to screen instead piping it to logile', action='store_true')
    config_parser.add_argument('--separate-lib64', help='Do not create symbolic links of files from lib64 in lib', action='store_true')
    config_parser.add_argument('--disable-shared', help='Disable building of shared libraries', action='store_true')
-    config_parser.add_argument('--all', help='Install all libraries with config file in config/', action='store_true')
-    config_parser.add_argument('--ignore-deps', help='Do not add dependencies of libraries to the install process', action='store_true')
-
+    
    parser.add_argument('--config', help='Path to config directory [$pwd/config]', default=os.getcwd()+"/config")
    parser.add_argument('--prefix', help='Path where install directory should be generated [$pwd]', default=os.getcwd())
    parser.add_argument('--src', help='Path where to download source  code to [$pwd/src]', default=os.getcwd()+"/src")
@ -39,12 +37,10 @@ def init():
    parser.add_argument('--compiler', help='Select compiler (gnu, intel, aocc) [gnu]', default='gnu')
    parser.add_argument('--mpi', help='Select compiler (hpcx, intelmpi, openmpi) [hpcx]', default='hpcx')
    parser.add_argument('--threads', help='Number of threads used for make [8]', default='8')
-    parser.add_argument('--verbose', help='Print build output to screen instead piping it to logfile', action='store_true')
+    parser.add_argument('--verbose', help='Print build output to screen instead piping it to logile', action='store_true')
    parser.add_argument('--separate-lib64', help='Do not create symbolic links of files from lib64 in lib', action='store_true')
    parser.add_argument('--disable-shared', help='Disable building of shared libraries', action='store_true')
-    parser.add_argument('--all', help='Install all libraries with config file in config/', action='store_true')
-    parser.add_argument('--ignore-deps', help='Do not add dependencies of libraries to the install process', action='store_true')
-
+    
    # run config parser and search config/*.json to add a build and version argument for it to the full parser
    config_dir = config_parser.parse_known_args()[0].config
    for cf in glob.glob(config_dir+"/*.json"):
@ -56,5 +52,5 @@ def init():
    # run full parser
    arg_namespace = parser.parse_args()
    args = vars(arg_namespace)
-
-    return arg_namespace, args
+    
+    return arg_namespace, args
--- a/lib/installer.py
+++ b/lib/installer.py
@ -32,8 +32,8 @@ def install_lib(lib, src_dir, work_dir, inst_dir, comp_cc, comp_cxx, comp_fc, bu
    if not os.path.exists(lib['name']+"-"+lib['version']):

        print("Downloading Source Code")
-        load_command = lib['download'].replace("$VERSION_UNDERSCORE", lib_version.replace(".", "_")).replace("$VERSION", lib_version)
-        err = subprocess.call(load_command, stdout=logfile, stderr=logfile, shell=True)
+        load_command = lib['download'].replace("$VERSION_UNDERSCORE", lib_version.replace(".", "_")).replace("$VERSION", lib_version).split()
+        err = subprocess.call(load_command, stdout=logfile, stderr=logfile)
        if err != 0:
            print("Error: Download of "+lib_name+" v."+lib_version+" failed!")
            print("See "+logfile_path+" for details")
@ -52,11 +52,6 @@ def install_lib(lib, src_dir, work_dir, inst_dir, comp_cc, comp_cxx, comp_fc, bu
        shared_option = ""
    config_command = lib['configure'].replace("$PREFIX", inst_dir).replace("$SHARED",shared_option)

-    # TODO: THIS IS UGLY AND NEEDS TO BE MOVED TO EITHER LIB OR COMPILER CONFIGURATION
-    # workaround for bad autotools reconization of flang
-    if (lib_name == "hdf5" or lib_name == "netcdf-f") and "aocc" in inst_dir:
-        config_command = config_command + " FCFLAGS=-fPIC"
-
    if verbose:
        print(underlined("\nConfiguring Library"))
    else:
@ -67,11 +62,6 @@ def install_lib(lib, src_dir, work_dir, inst_dir, comp_cc, comp_cxx, comp_fc, bu
        print("See "+logfile_path+" for details")
        sys.exit(1)

-    # TODO: THIS IS UGLY AND NEEDS TO BE MOVED TO EITHER LIB OR COMPILER CONFIGURATION
-    # workaround for bad autotools reconization of flang
-    if (lib_name == "hdf5" or lib_name == "netcdf-f") and "aocc" in inst_dir:
-        err = subprocess.call("sed -i -e 's/wl=\"\"/wl=\"-Wl,\"/g' libtool", shell=True)
-
    # build library
    build_command = lib['build'].replace("$BUILDTHREADS", build_threads)
    if verbose:
@ -83,6 +73,9 @@ def install_lib(lib, src_dir, work_dir, inst_dir, comp_cc, comp_cxx, comp_fc, bu
        finished = False
        while build_task.poll() is None:
            if not finished:
+                # bar_width = num_objects
+                # while (bar_width > 100):
+                #    bar_width = int(bar_width/10)
                bar_width = 50
                for i in progressbar(range(num_objects),  "Building library: ", bar_width):
                    while len(glob.glob('**/*.o', recursive=True)) < i:
@ -110,10 +103,7 @@ def install_lib(lib, src_dir, work_dir, inst_dir, comp_cc, comp_cxx, comp_fc, bu
            print("See "+logfile_path+" for details")
            sys.exit(1)
    print("Library "+lib_name+" has been installed successfully!\n")
-
-    # copy logfile to install location
-    if not verbose:
-        build_output_path = inst_dir+"/"+"build_info"
-        if not os.path.exists(build_output_path):
-                os.makedirs(build_output_path)
-        shutil.copyfile(logfile_path, build_output_path+"/"+lib_name+".log")
+    build_output_path = inst_dir+"/"+"build_info"
+    if not os.path.exists(build_output_path):
+            os.makedirs(build_output_path)
+    shutil.copyfile(logfile_path, build_output_path+"/"+lib_name+".log")
--- a/lib/io.py
+++ b/lib/io.py
@ -1,8 +0,0 @@
-import json
-
-
-# load library data from json file
-def load_lib_data(lib_path):
-    with open (lib_path, 'r') as config_file:
-        data = json.load(config_file)
-        return data
--- a/lib/sort.py
+++ b/lib/sort.py
@ -25,24 +25,14 @@ def sort_libs_by_dependencies(selected_libs):
    if len(selected_libs) < 2:
        return selected_libs

-    lib_names = []
-    for lib in selected_libs:
-        lib_names.append(lib['name'])
-
    deplist = []
    for lib in selected_libs:
        name = lib['name']
-        # only sort after dependencies that are actually present to avoid cyclic dependencies
-        deps = []
-        for dep in lib['dependencies'].split(','):
-            if dep in lib_names:
-                deps.append(dep)
-        dependencies = set(deps)
+        dependencies = set(lib['dependencies'].split(','))
        if dependencies == {''}:
            dependencies = {}
        deplist.append([name, dependencies])
    sorted_deplist = topological_sort(deplist)
-
    sorted = []
    for entry in sorted_deplist:
        for lib in selected_libs:
--- a/lib/toolchain.py
+++ b/lib/toolchain.py
@ -1,22 +1,15 @@
 import os
-import sys
 from lib.shell import get_from_command


 def get_compiler_version(compiler):
    if compiler == "gnu":
        cc = "gcc"
-        compstr = get_from_command([cc, "-dumpversion"]).strip()
    elif compiler == "intel":
        cc = "icc"
-        compstr = get_from_command([cc, "-dumpversion"]).strip()
    elif compiler == "aocc":
        cc = "clang"
-        compstr = get_from_command([cc, "-dumpversion"]).strip()
-    elif compiler == "nec":
-        cc = "ncc"
-        rawstr = get_from_command([cc, "--version"])
-        compstr = rawstr.split()[2]
+    compstr = get_from_command([cc, "-dumpversion"]).strip()
    return compstr


@ -31,14 +24,11 @@ def get_mpi_version(mpi):
        rawstr = get_from_command(["ompi_info"]).splitlines()[1]
        mpistr = rawstr[rawstr.find(":")+2:]
    elif mpi == "intelmpi":
-        rawstr = get_from_command(["mpirun","--version"])
-        mpistr = rawstr.split()[7]
+        rawstr = get_from_command(["ompi_info"]).splitlines()[0]
+        mpistr = rawstr[rawstr.find("Version")+8:rawstr.find("Build")-1]
    elif mpi == "mpich":
        rawstr = get_from_command(["mpirun", "--version"])
        mpistr = rawstr.split()[4]
-    elif mpi == "necmpi":
-        rawstr = get_from_command(["mpirun", "--version"])
-        mpistr = rawstr.split()[4]
    return mpistr


@ -48,17 +38,10 @@ def set_toolchain(compiler, mpi):
            cc = "mpicc"
            cxx = "mpic++"
            fc = "mpifort"
-        elif mpi == "intelmpi":
+        elif mpi == "intel":
            cc = "mpicc"
            cxx = "mpicxx"
            fc = "mpif90"
-        elif mpi == "necmpi":
-            cc = "mpincc -vh"
-            cxx = "mpinc++ -vh"
-            fc = "mpinfort -vh"
-            os.environ["NMPI_CC_H"] = "gcc"
-            os.environ["NMPI_CXX_H"] = "g++"
-            os.environ["NMPI_FC_H"] = "gfortran"
    elif compiler == "aocc":
        if mpi == "hpcx" or mpi == "openmpi":
            cc = "mpicc"
@ -69,17 +52,10 @@ def set_toolchain(compiler, mpi):
            os.environ["OMPI_MPIFC"] = "flang"
            os.environ["OMPI_MPIF90"] = "flang"
            os.environ["OMPI_MPIF77"] = "flang"
-        elif (mpi == "intelmpi"):
-            cc = "mpicc -cc=clang"
-            cxx = "mpicxx -cxx=clang++"
-            fc = "mpif90 -fc=flang"
-        elif mpi == "necmpi":
-            cc = "mpincc -vh"
-            cxx = "mpic++ -vh"
-            fc = "mpinfort -vh"
-            os.environ["NMPI_CC_H"] = "clang"
-            os.environ["NMPI_CXX_H"] = "clang++"
-            os.environ["NMPI_FC_H"] = "flang"
+        elif (mpi == "intel"):
+            cc = "\"mpicc -cc=clang\""
+            cxx = "\"mpicxx -cxx=clang++\""
+            fc = "\"mpif90 -fc=flang\""
    elif compiler == "intel":
        if mpi == "hpcx" or mpi == "openmpi":
            cc = "mpicc"
@ -90,23 +66,11 @@ def set_toolchain(compiler, mpi):
            os.environ["OMPI_MPIFC"] = "ifort"
            os.environ["OMPI_MPIF90"] = "ifort"
            os.environ["OMPI_MPIF77"] = "ifort"
-        elif mpi == "intelmpi":
+        elif mpi == "intel":
            cc = "mpiicc"
            cxx = "mpiicpc"
            fc = "mpiifort"
-        elif mpi == "necmpi":
-            cc = "mpincc -vh"
-            cxx = "mpic++ -vh"
-            fc = "mpinfort -vh"
-            os.environ["NMPI_CC_H"] = "icc"
-            os.environ["NMPI_CXX_H"] = "icpc"
-            os.environ["NMPI_FC_H"] = "ifort"
-    elif compiler == "nec":
-        # only NEC MPI
-        cc = "mpincc"
-        cxx = "mpinc++"
-        fc = "mpinfort"
-
+ 
    # set environment variables
    os.environ["CC"] = cc
    os.environ["CXX"] = cxx
@ -114,4 +78,4 @@ def set_toolchain(compiler, mpi):
    os.environ["F90"] = fc
    os.environ["F77"] = fc

-    return cc, cxx, fc
+    return cc, cxx, fc
--- a/29
+++ b/29
@ -1,14 +1,16 @@
 #!/usr/bin/env python3
 import shutil
+import json
 import os
 import glob
-from lib.ui import bordered, print_welcome
+from lib.ui import progressbar, bordered, underlined, print_welcome
+from lib.shell import get_from_command
 from lib.toolchain import get_compiler_version, get_mpi_version, set_toolchain
-from lib.dependency import sort_libs_by_dependencies, load_selected_libs
+from lib.sort import sort_libs_by_dependencies
 from lib.installer import install_lib
 from lib.init import init, check_python_version

-SCRIPT_VERSION = "v1.0"
+SCRIPT_VERSION = "v0.6"

 # check if Python >=3.3.0 is used
 check_python_version()
@ -28,11 +30,20 @@ build_threads = arg_namespace.threads
 verbose = arg_namespace.verbose
 separate_lib64 = arg_namespace.separate_lib64
 disable_shared = arg_namespace.disable_shared
-install_all_libs = arg_namespace.all
-ignore_deps = arg_namespace.ignore_deps

 # extract libraries and versions selected for installation
-selected_libs = load_selected_libs(config_dir, arg_namespace, args, install_all_libs, ignore_deps)
+selected_libs = []
+ignore_names = ["config", "mpi", "compiler", "prefix", "src", "work", "threads", "verbose", "version", "disable_shared"]
+for lib_name in args:
+    if lib_name not in ignore_names and "version" not in lib_name:
+        install = getattr(arg_namespace, lib_name)
+        if install:
+            version = getattr(arg_namespace, lib_name+"_version")
+            config_file = config_dir + "/" + lib_name + ".json"
+            with open(config_file, 'r') as cf:
+                data = json.load(cf)
+                data['version'] = version
+                selected_libs.append(data)

 # set up install directory name
 compiler_version = get_compiler_version(compiler)
@ -51,10 +62,6 @@ print_welcome(SCRIPT_VERSION, compiler, compiler_version, mpi, mpi_version, buil

 # install selected libraries
 if len(sorted_libs) > 0:
-    # add install dir to environment
-    os.environ["LIBRARY_PATH"] = inst_dir + "/lib" + os.pathsep + inst_dir + "/lib64" + os.pathsep + os.environ["LIBRARY_PATH"]
-    os.environ["LD_LIBRARY_PATH"] = inst_dir + "/lib" + os.pathsep + inst_dir + "/lib64" + os.pathsep + os.environ["LD_LIBRARY_PATH"]
-    # install libraries
    for lib in sorted_libs:
        install_lib(lib, src_dir, work_dir, inst_dir, cc, cxx, fc, build_threads, disable_shared, verbose)
    print(bordered("ALL INSTALLS COMPLETED SUCCESSFULLY\nPlease add "+inst_dir+" to your environment"))
@ -77,4 +84,4 @@ if (not keep_work and os.path.exists(work_dir)):
    shutil.rmtree(work_dir)
    print("Done.")

-print("\n")
+print("\n")
--- a/tests/init.py
+++ b/tests/init.py
--- a/tests/test.py
+++ b/tests/test.py
@ -0,0 +1,17 @@
+from lib.sort import sort_libs_by_dependencies
+from lib.shell import get_from_command
+
+
+def test_sort():
+    lib_a = {"name":"lib_A", "dependencies":""}
+    lib_b = {"name":"lib_B", "dependencies":"lib_A"}
+    lib_c = {"name":"lib_C", "dependencies":"lib_B"}
+    libs = [lib_c, lib_b, lib_a]
+    sorted = [lib_a, lib_b, lib_c]
+    assert sort_libs_by_dependencies(libs) == sorted
+
+
+def test_shell():
+    assert get_from_command(["echo", "Hello", "World"]) == "Hello World\n"
+
+