valyala/fasthttp
View on GitHubfasthttp client. Read body in chunks and stop downloading on condition.
Open
#1,065 opened on Aug 5, 2021
help wanted
Repository metrics
- Stars
- (21,741 stars)
- PR merge metrics
- (Avg merge 1d 11h) (34 merged PRs in 30d)
Description
For example, I want to stop downloading content if it has wrong mime type. How can I do so?
The documentation shows only resp.Body() which gets whole response.
Below is how I use standard http library:
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body := resp.Body
defer body.Close()
checkedMime := false
readBuffer := make([]byte, 1 << 12)
bodyData := make([]byte, 0, typicalSize)
for {
n, err := body.Read(readBuffer)
if err != nil && err != io.EOF {
return nil, err
}
bodyData = append(bodyData, readBuffer[0:n]...)
if !checkedMime && len(bodyData) > 512{
mimeType := http.DetectContentType(bodyData)
if isArchive(mimeType){
isArchive = true
break
}
if mimeType != "text/html"{
isUselessContent = true
break
}
checkedMime = true
}
if len(bodyData) > tooBigContentSize {
tooBigContent = true
break
}
if err == io.EOF {
break
}
}
// do something with bodyData, isArchive, isUselessContent, tooBigContent
...