Building NSIS Installers for Large File Distributions

I've been working on some solutions recently to distribute large data sets utilizing numerous compressed files groups. I decided the best way to dummy proof this was to wrap an installer around them and do it "right". So here is how to do that with an installer.

If you need to install, with only one setup application, two or more tar, bz2, gz, or lzma compressed files (for example multiple clustered files of over 2GB containing scientific data for your application and a couple others containing the app, and maybe a required piece of library software like winpcap) you need a robust solution such as the Nullsoft Install System - NSIS.

The most logical idea is to create a single file, but NSIS does have file size limitations within it's compiler. Currently it is about 2GB in size. So deploying a package of say 8GB (something that might normally fit on a Dual Layer DVD) is not possible with standard NSIS single file installers. This solution uses external plugins to decompress the files within the same directory framework as the installer. This allows you to create large file distributions that could be delivered on large media or across gigabit speed networks.

Tools Req:

7zip [installer] - Compression Utility
Notepad++ [installer] - IDE
NSIS [installer]
UltraModernUI NSIS User Interface [installer] - personal choice of GUI for NSIS installer
Untgz Contrib plugin [installer] - Decompression library

Files to Distrubute:
compressed_1.tar
-- decomp_set1of5_file1of2.txt
-- decomp_set1of5_file2of2.txt
compressed_2.tar
-- decomp_set2of5_file1of3.txt
-- decomp_set2of5_file2of3.txt
-- decomp_set2of5_file3of3.txt
compressed_3.tar
-- decomp_set3of5_file1of2.txt
-- decomp_set3of5_file2of2.txt
compressed_4.tar
-- decomp_set4of5_file1of1.txt
compressed_5.tar
-- decomp_set5of5_file1of3.txt
-- decomp_set5of5_file2of3.txt
-- decomp_set5of5_file3of3.txt

1
2
3 !include LogicLib.nsh
4
5 Function .onInit
6 # Section Size must be manually set to the size of the required disk space NSIS will not do this for external files.
7 # set required size of section number of kilobytes
8 # 8gb to kilo bytes = 8,388,608
9 SectionSetSize ${SecDecompress} 8388608
10
11;compressed_#.taz has be in the same directory as the Setup file.
12${If} ${FileExists} "$EXEDIR\compressed_1.tar"
13${AndIf} ${FileExists} "$EXEDIR\compressed_2.tar"
14${AndIf} ${FileExists} "$EXEDIR\compressed_3.tar"
15 ${AndIf} ${FileExists} "$EXEDIR\compressed_4.tar"
16 ${AndIf} ${FileExists} "$EXEDIR\compressed_5.tar"
17 Return
18 ${Else}
19 MessageBox MB_OK|MB_ICONINFORMATION "This copy of the installer is missing a compressed#.tar file.." IDOK abort
20 abort:
21 Banner::destroy
22 Abort
23 ${EndIf}
24
25 FunctionEnd
26
27 Section -decompress SecDecompress
28
29 ;UnTGZ Plugin
30 ;compressed_#.tar in this example is not compressed by gzip just tar collection
31 ; untgz plugin requires -znone to denote this
32
33 untgz::extract -j -d "$INSTDIR\" -znone"$EXEDIR\compressed_1.tar"
34 ${If}${FileExists} "$INSTDIR\decomp_set1of5_file1of2.txt"
35 ${AndIf} ${FileExists} "$INSTDIR\decomp_set1of5_file2of2.txt"
36 untgz::extract -j -d "$INSTDIR\" -znone"$EXEDIR\compressed_2.tar"
37 ${AndIf} ${FileExists} "$INSTDIR\decomp_set2of5_file1of3.txt"
38 ${AndIf} ${FileExists} "$INSTDIR\decomp_set2of5_file2of3.txt"
39 ${AndIf} ${FileExists} "$INSTDIR\decomp_set2of5_file3of3.txt"
40 untgz::extract -j -d "$INSTDIR\" -znone"$EXEDIR\compressed_3.tar"
41 ${AndIf} ${FileExists} "$INSTDIR\decomp_set3of5_file1of2.txt"
42 ${AndIf} ${FileExists} "$INSTDIR\decomp_set3of5_file2of2.txt"
43 untgz::extract -j -d "$INSTDIR\" -znone"$EXEDIR\compressed_4.tar"
44 ${AndIf} ${FileExists} "$INSTDIR\decomp_set4of5_file1of1.txt"
45 untgz::extract -j -d "$INSTDIR\" -znone"$EXEDIR\compressed_5.tar"
46 ${AndIf} ${FileExists} "$INSTDIR\decomp_set5of5_file1of3.txt"
47 ${AndIf} ${FileExists} "$INSTDIR\decomp_set5of5_file2of3.txt"
48 ${AndIf} ${FileExists} "$INSTDIR\decomp_set5of5_file3of3.txt"
49 Goto EverythingOk
50 ${Else}
51 MessageBox MB_OK|MB_ICONEXCLAMATION "Installation Failure. Media may be corrupt." IDOKabort
52 abort:
53 Banner::destroy
54 Abort
55 ${EndIf}
56 EverythingOK:
57
58 ;If tar files were packaged into the setup you can delete it like this :)
59 ;Delete "$INSTDIR\compressed#.taz"
60
61 SectionEnd