[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linux-crypto-vger
Subject:    Re: PATCH crypto/zlib: Allow Z_OK return value from zlib_deflate
From:       Herbert Xu <herbert () gondor ! apana ! org ! au>
Date:       2010-01-13 9:45:04
Message-ID: 20100113094504.GA11422 () gondor ! apana ! org ! au
[Download RAW message or body]

On Wed, Dec 23, 2009 at 08:01:57PM +0800, Herbert Xu wrote:
>
> Hmm, it seems that the semantics of the final function isn't
> quite right.  It was never meant to be called multiple times.
> 
> Let me see if we can fix this up without touching the interface.

How about this patch? So now the final functions are more like
their zlib counterparts.  If it runs out of output buffer, it
returns avail_out + 1 and you need to call it again.

diff --git a/crypto/zlib.c b/crypto/zlib.c
index c301573..632b4b4 100644
--- a/crypto/zlib.c
+++ b/crypto/zlib.c
@@ -190,19 +190,23 @@ static int zlib_compress_final(struct crypto_pcomp *tfm,
 	stream->avail_out = req->avail_out;
 
 	ret = zlib_deflate(stream, Z_FINISH);
+	if (ret == Z_OK)
+		ret = req->avail_out + 1;
 	if (ret != Z_STREAM_END) {
 		pr_debug("zlib_deflate failed %d\n", ret);
 		return -EINVAL;
-	}
+	} else
+		ret = req->avail_out - stream->avail_out;
 
-	ret = req->avail_out - stream->avail_out;
 	pr_debug("avail_in %u, avail_out %u (consumed %u, produced %u)\n",
 		 stream->avail_in, stream->avail_out,
 		 req->avail_in - stream->avail_in, ret);
+
 	req->next_in = stream->next_in;
 	req->avail_in = stream->avail_in;
 	req->next_out = stream->next_out;
 	req->avail_out = stream->avail_out;
+
 	return ret;
 }
 
@@ -323,15 +327,19 @@ static int zlib_decompress_final(struct crypto_pcomp *tfm,
 		}
 	} else
 		ret = zlib_inflate(stream, Z_FINISH);
-	if (ret != Z_STREAM_END) {
+
+	if (ret == Z_OK)
+		ret = req->avail_out + 1;
+	else if (ret != Z_STREAM_END) {
 		pr_debug("zlib_inflate failed %d\n", ret);
 		return -EINVAL;
-	}
+	} else
+		ret = req->avail_out - stream->avail_out;
 
-	ret = req->avail_out - stream->avail_out;
 	pr_debug("avail_in %u, avail_out %u (consumed %u, produced %u)\n",
 		 stream->avail_in, stream->avail_out,
 		 req->avail_in - stream->avail_in, ret);
+
 	req->next_in = stream->next_in;
 	req->avail_in = stream->avail_in;
 	req->next_out = stream->next_out;

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
[prev in list] [next in list] [prev in thread] [next in thread]