The Shed
Check-in [d516e7425d]
Login
HomeTimelineFilesBranchesTagsTicketsWiki
Overview
Comment:Decompressor might loose part of the underlying buffer array by reslicing, fixed by copy
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: d516e7425df001417d5f191f0fd88196f3867aa9
User & Date: spaskalev on 2014-12-20 13:04:59
Other Links: manifest | tags
Context
2014-12-20
17:44
Buffer the input on decompressing, not the decompressor itself. This takes pdc -d < linux-3.18.1.tar.pdc > linux-3.18.1.tar down to 11 seconds from 13 minutes :) check-in: 4f0d26907d user: spaskalev tags: trunk
13:04
Decompressor might loose part of the underlying buffer array by reslicing, fixed by copy check-in: d516e7425d user: spaskalev tags: trunk
11:52
[predictor] Removed the buffer from the context struct, allocate the input slice buffer on creation with make. check-in: 723ffeb1fd user: spaskalev tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Modified src/0dev.org/predictor/predictor.go from [b8ee77a9c4] to [90c8b12e57].

︙︙
134
135
136
137
138
139
140




141

142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164

		if len(output) == 0 {
			return 0, nil
		}

		// Check whether we have leftover data in the buffer
		if len(ctx.input) > 0 {
			readCount = copy(output, ctx.input)




			ctx.input = ctx.input[readCount:]

			return readCount, nil
		}

		// This is single-iteration only but it is fine according to io.Reader's contract ?!
		// TODO - read all bytes from a block based on the hamming weight of the flag
		// and just shuffle them for predictions instead of bite-sized reads ;)

		// Read the flags
		ctx.input = ctx.input[:1]
		readCount, err = wrapped.Read(ctx.input)
		if readCount == 0 || err != nil {
			return readCount, err
		}

		flags = ctx.input[0]
		ctx.input = ctx.input[:8]

		var i uint = 0
		for ; i < 8; i++ {
			if flags&(1<<i) > 0 {
				// Guess was right
				ctx.input[i] = ctx.table[ctx.hash]
			} else {








>
>
>
>
|
>








<
|




|
|








134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154

155
156
157
158
159
160
161
162
163
164
165
166
167
168

		if len(output) == 0 {
			return 0, nil
		}

		// Check whether we have leftover data in the buffer
		if len(ctx.input) > 0 {
			readCount = copy(output, ctx.input)

			// Check whether we still have leftover data in the buffer :)
			if readCount < len(ctx.input) {
				copy(ctx.input[:readCount], ctx.input[readCount:])
				ctx.input = ctx.input[:readCount]
			}
			return readCount, nil
		}

		// This is single-iteration only but it is fine according to io.Reader's contract ?!
		// TODO - read all bytes from a block based on the hamming weight of the flag
		// and just shuffle them for predictions instead of bite-sized reads ;)

		// Read the flags

		readCount, err = wrapped.Read(ctx.input[:1])
		if readCount == 0 || err != nil {
			return readCount, err
		}

		ctx.input = ctx.input[:8]
		flags = ctx.input[0]

		var i uint = 0
		for ; i < 8; i++ {
			if flags&(1<<i) > 0 {
				// Guess was right
				ctx.input[i] = ctx.table[ctx.hash]
			} else {
︙︙

Powered by Fossil · RSS