From 6b87843fc38c1646615ab0342a703f7ab3caf1cb Mon Sep 17 00:00:00 2001 From: Mohammad Akhlaghi Date: Thu, 26 Nov 2020 03:45:54 +0000 Subject: The long about.hml is now broken up into smaller pages The "About" page ('about.html') was effectively a full copy of Maneage's 'README-hacking.md', so it was very long. To help in readability it has now been broken down into smaller pages (one for each section). Also the indentation of Make recipes was corrected, both in the about pages, and also in the tutorial. --- about-future.html | 160 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 about-future.html (limited to 'about-future.html') diff --git a/about-future.html b/about-future.html new file mode 100644 index 0000000..a8ef98c --- /dev/null +++ b/about-future.html @@ -0,0 +1,160 @@ + + + + + + + + + Maneage -- Managing data lineage + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ + + + + +
+

Previous: Tips for designing your project, Up: About

+ +

Future improvements

+ +

This is an evolving project and as time goes on, it will evolve and become + more robust. Some of the most prominent issues we plan to implement in the + future are listed below, please join us if you are interested.

+ +

Package management

+ +

It is important to have control of the environment of the project. Maneage + currently builds the higher-level programs (for example GNU Bash, GNU Make, + GNU AWK and domain-specific software) it needs, then sets PATH so the + analysis is done only with the project's built software. But currently the + configuration of each program is in the Makefile rules that build it. This + is not good because a change in the build configuration does not + automatically cause a re-build. Also, each separate project on a system + needs to have its own built tools (that can waste a lot of space).

+ +

A good solution is based on the Nix package manager: a separate file is present for + each software, containing all the necessary info to build it (including its + URL, its tarball MD5 hash, dependencies, configuration parameters, build + steps and etc). Using this file, a script can automatically generate the + Make rules to download, build and install program and its dependencies + (along with the dependencies of those dependencies and etc).

+ +

All the software are installed in a "store". Each installed file (library + or executable) is prefixed by a hash of this configuration (and the OS + architecture) and the standard program name. For example (from the Nix + webpage):

+ + 
/nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1/
+ +

The important thing is that the "store" is not in the project's search + path. After the complete installation of the software, symbolic links are + made to populate each project's program and library search paths without a + hash. This hash will be unique to that particular software and its + particular configuration. So simply by searching for this hash in the + installed directory, we can find the installed files of that software to + generate the links.

+ +

This scenario has several advantages: 1) a change in a software's build + configuration triggers a rebuild. 2) a single "store" can be used in many + projects, thus saving space and configuration time for new projects (that + commonly have large overlaps in lower-level programs).

+ + +

Previous: Tips for designing your project, Up: About

+ + + + + +
+ -- cgit v1.2.1