[ISSUE] Install local compiler fails always (gcc-arm)

ISSUE created form my initial post

and the suggestion of @m_m

Since I think a have a reliable network (download via browser works, no issue on other places).

To get some more insights I debugged the particle.particle-vscode-core-1.0.0-alpha.3 extensions with code. I see that it is the checksum check inside DependencyManager.downloadDependency that always fails

if (hash !== dependency.sha256){
	// TODO (mirande): delete file?
	throw new Error('file hashes do not match!');
}

I then added a log output to this

if (hash !== dependency.sha256){
	// TODO (mirande): delete file?
	console.log(`**-> hashes do not match exp=${dependency.sha256} != act=${hash}`, dependency);
	throw new Error('file hashes do not match!');
}

And actually see that the hash of the file is each time different! I also see that the tmpFile in the filesystem, have a different size each time and always smaller than the size of the original file.

I then added another log output to the reportDependencyProgress

function reportDependencyProgress(status){
	let msgMap = {};

	return (dependency, progress) => {
		msgMap[dependency.id] = `${progress.percentage.toFixed(2)}%`;
		const text = Object.keys(msgMap).sort().map(key => `${key}: ${msgMap[key]}`);	
		status.text = text.join(' | ');
		console.log("**-> ", status.text);
		status.show();
	};
}

When I now run the install I see that the gcc-arm % increases at a steady speed but somewhere around 60% (always a bit at a different %) suddenly jumps to 100%.

**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 55.91%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 56.16%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 56.43%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 56.70%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 56.96%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 57.22%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 57.49%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 57.75%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 58.01%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 58.28%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 58.54%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 58.81%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 59.08%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 59.34%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 59.58%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 59.61%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 60.66%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 60.94%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 61.20%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 61.46%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 61.73%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 61.99%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 62.06%
**->  buildscripts@1.0.0: 100.00% | buildtools@1.0.0: 100.00% | deviceOS@0.7.0: 100.00% | gcc-arm@5.3.1: 100.00%
**-> hashes do not match exp=f41f32436188c3017422635b187378c8bf0938a3ad396506b2d3b8ce1fb402c9 != act=e776b5f41304ff2bee9883ff131b0b8809abd7c31e8e50e36b1e45113809d193
undefined
Object
id
"gcc-arm@5.3.1"
name
"gcc-arm"
version
"5.3.1"
main
"./bin"
url
"https://binaries.particle.io/gcc-arm/darwin/x64/gcc-arm-v5.3.1.tar.gz"
sha256
"f41f32436188c3017422635b187378c8bf0938a3ad396506b2d3b8ce1fb402c9"
root
"/Users/bittailor/.particle/toolchains/gcc-arm/5.3.1"
dir
"/Users/bittailor/.particle/toolchains/gcc-arm/5.3.1/bin"

So I think somehow the download stops before the whole file is downloaded and then of course the checksum/hash does not match.

I tried to understand the downloading logic. I saw that it is mainly a node source stream that is piped into a file stream but was not yet able to grasp all the additional logic of trackProgressMaybe and why a pipe could finish too early without emitting and error

		srcStream.on('error', err => reject(err));
		destStream.on('error', err => reject(err));
		destStream.on('finish', () => resolve({ filename: destination }));
		trackProgressMaybe(srcStream, onProgress, interval, size).pipe(destStream);

Supporting information

  • VSCode Version: 1.27.2 (1.27.2)
  • OS Version: macOS Sierra 10.12.6 (16G1510)
  • Other extensions installed: many (39)

Steps to Reproduce:

  1. Press install on the Please install the Particle Local Compiler

When I download the gcc-arm-v5.3.1.tar.gz via wget I see that the server closes the connection once before the the whole file is downloaded and wget needs one reconnect and continue the download.

$ wget https://binaries.particle.io/gcc-arm/darwin/x64/gcc-arm-v5.3.1.tar.gz
--2018-10-06 17:28:20--  https://binaries.particle.io/gcc-arm/darwin/x64/gcc-arm-v5.3.1.tar.gz
Resolving binaries.particle.io... 195.186.210.161
Connecting to binaries.particle.io|195.186.210.161|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 123864745 (118M) [application/x-gzip]
Saving to: 'gcc-arm-v5.3.1.tar.gz.1'

gcc-arm-v5.3.1.tar.gz.1              77%[====================================================>                 ]  90.97M  3.07MB/s   in 30s

2018-10-06 17:28:51 (2.98 MB/s) - Connection closed at byte 95394059. Retrying.

--2018-10-06 17:28:52--  (try: 2)  https://binaries.particle.io/gcc-arm/darwin/x64/gcc-arm-v5.3.1.tar.gz
Connecting to binaries.particle.io|195.186.210.161|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 123864745 (118M), 28470686 (27M) remaining [application/x-gzip]
Saving to: 'gcc-arm-v5.3.1.tar.gz.1'

gcc-arm-v5.3.1.tar.gz.1             100%[+++++++++++++++++++++++++++++++++++++++++++++++++++++================>] 118.13M  3.07MB/s   in 8.9s

2018-10-06 17:29:01 (3.03 MB/s) - 'gcc-arm-v5.3.1.tar.gz.1' saved [123864745/123864745]

I think the download logic in the DependencyManager does not implement this reconnect and continue and therefore stops and fails on the first close of the connection.

Thanks for the details, @bittailor. We’ll try to see what’s going on.

Can you see if you have anything in ~/.particle/toolchains/gcc-arm/?

I do not even have a toolchains folder inside ~/.particle.

I quickly hacked a simple and incomplete wget for node that does additional range requests if the connection is closed before the whole file is streamed. I then changed DependencyManager.downloadDependency form

	async downloadDependency(dependency, onProgress){
		const { writeFileStream } = fsUtils;

		try {
			const tmpFile = await tmp.createFile();
						
			const res = await fetchDependencyBundle(dependency, this.proxyCfg); // TODO (mirande): retry on error?
			const size = parseInt(res.headers.get('content-length'), 10);
			const { hash } = await writeFileStream(res.body, tmpFile.path, { onProgress, size });
			
			if (hash !== dependency.sha256){
				// TODO (mirande): delete file?
				throw new Error('file hashes do not match!');
			}
			dependency.tmpFile = tmpFile;
		} catch (error){
			throw new DependencyManagerDownloadError(dependency, error);
		}
		return dependency;
	}

to

	async downloadDependency(dependency, onProgress){
		const { writeFileStream } = fsUtils;

		try {
			const tmpFile = await tmp.createFile();
			
			const { hash } = await wget(dependency.url, tmpFile.path, onProgress);

			if (hash !== dependency.sha256){
				// TODO (mirande): delete file?
				throw new Error('file hashes do not match!');
			}
			dependency.tmpFile = tmpFile;
		} catch (error){
			throw new DependencyManagerDownloadError(dependency, error);
		}
		return dependency;
	}

With this I was able to install the local compiler and now have the folder ~/.particle/toolchains and also stuff inside ~/.particle/toolchains/gcc-arm/.

So it looks like it is an issue that in my environment the gcc-arm-v5.3.1.tar.gz can not be downlaoded by a single HTTP get request.

2 Likes

Thanks for continuing to provide details! Is there anything unique about your environment that might be impacting downloads? Maybe a daemon or virtual networking device? We’ll investigate our download logic but it would be good to understand why we haven’t seen this issue during our testing with a similar setup.

No not something I would be aware of.

  • I tried the download from two different Macs in my home network (internet provider www.swisscom.ch) same behavior.
  • I also tried on another home network (same internet provider) same behavior.
  • I tried from a azure vm (Location West Europe) other behavior (connection not closed during download).
  • I tried from my server of one of my web hosting provider other behavior (connection not closed during download).

What might be interesting is that binaries.particle.io is resolved to different IP's depending from where the wget is executed:

  • When executed in a network connected to my internet provider: Resolving binaries.particle.io ... 195.186.210.161
  • When executed on azure vm (Location West Europe): Resolving binaries.particle.io ... 54.192.129.147, 54.192.129.124, 54.192.129.176, ...
  • When executed on web hosting provider's server: Resolving binaries.particle.io ... 13.32.28.175, 13.32.28.244, 13.32.28.140, ...
1 Like

My issue still exists in particle.particle-vscode-core-1.0.0-alpha.5.

I again had to apply my patch to DependencyManager.downloadDependency be able to successfully run Install Local Compiler.

My issue still exists in particle.particle-vscode-core-1.0.0-alpha.6.

I again had to apply my patch to DependencyManager.downloadDependency be able to successfully run Install Local Compiler.

My issue still exists in particle.particle-vscode-core-1.0.0-alpha.7.

I again had to patch DependencyManager.downloadDependency be able to successfully run Install Local Compiler.

hey @bittailor :wave:

i’ve overhauled the downloader - would you be willing to test a special build of Workbench (core) and let us know if your situation is improved?

either way, thanks for these reports :pray:

Hi

Always willing to test and maybe help me and others :wink: .

Where would I find the special build of Workbench (core) to try it out.

much appreciated :pray: - i’ll DM you a link :+1:

Works perfect, the whole toolchain including compiler installs without a hickup :fireworks:.
Afterwards I could local compile, flash (and run a blinking LED) on a photon and a xenon.

So :+1::+1: and thanks.

1 Like

excellent! thanks for being patient and for trying Workbench :pray::+1: