求助, Java 接口上传 2G 以上大文件 EOFException: null

查看 84|回复 9
作者:shiyu6226   
最近在写一个 web 端的私人网盘服务,测试发现上传 2G 以上大文件时 后台会出现异常,请问有大佬做过相关的需求吗?怎么解决这类问题?
异常日志如下
ERROR 19593 --- [io-18073-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.web.multipart.MultipartException: Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.impl.IOFileUploadException: Processing of multipart/form-data request failed. java.io.EOFException] with root cause
java.io.EOFException: null
...
我修改了好多参数也不好使
@Configuration
@Slf4j
public class EmbeddedTomcatConfig implements WebServerFactoryCustomizer {
@Override
public void customize(ConfigurableServletWebServerFactory factory) {
    log.info("Init EmbeddedTomcatConfig...");
    ((TomcatServletWebServerFactory)factory).addConnectorCustomizers(new TomcatConnectorCustomizer() {
        @Override
        public void customize(Connector connector) {
            Http11NioProtocol protocol = (Http11NioProtocol) connector.getProtocolHandler();
            protocol.setMaxConnections(3000);
            protocol.setMaxThreads(800);
            protocol.setAcceptCount(200);
            protocol.setSelectorTimeout(30000);
            protocol.setSessionTimeout(60000 * 2);
            protocol.setConnectionTimeout(60000 * 5);
            protocol.setDisableUploadTimeout(false);
            protocol.setConnectionUploadTimeout(60000 * 10);
        }
    });
}
}
application 参数
spring.servlet.multipart.max-request-size=-1
spring.servlet.multipart.max-file-size=-1
server.tomcat.max-swallow-size=-1
server.tomcat.max-http-form-post-size=-1
控制层
@ResponseBody
@ApiOperation(value = "上传文件",notes = "上传文件")
@RequestMapping(value = "/FilesUpload",method = RequestMethod.POST)
public BaseResponse uploadFiles(
        @RequestParam(required = true) MultipartFile files,
        HttpServletRequest request,
        HttpServletResponse response
) {
    if (files.isEmpty() || files.getSize() == 0) {
        response.setStatus(HttpServletResponse.SC_EXPECTATION_FAILED);
        return BaseResponse.initErrorBaseResponse("不能上传空文件!");
    }
    try {
        return BaseResponse.initSuccessBaseResponse(fileExecuteService.uploadFiles(files,request), "操作成功");
    } catch (Exception e) {
        response.setStatus(HttpServletResponse.SC_EXPECTATION_FAILED);
        return BaseResponse.initErrorBaseResponse(e.getMessage());
    }
}
xiaohundun   
应该不是文件大小配置的问题,因为不是这个异常,这个异常应该看看是不是网络的问题
zsj1029   
大文件请用流式传输,普通文件操作 2g 会占用 2g 物理内存,大了会炸
Stendan   


koloonps   
不用流式传输就需要在客户端把文件切好分开上传,不然分分钟 OOM
shiyu6226
OP
  
@xiaohundun 这个是用在内网环境下的,网络应该是不受影响
shiyu6226
OP
  
@zsj1029
@koloonps
shiyu6226
OP
  
@zsj1029
@koloonps
实现方式是用的缓冲流写入的,实际运行过程中,内存使用一直保持在 500MB 上下,代码如下
private static boolean writeFileToLocal(String toLocalFilePath, MultipartFile file) throws Exception {
boolean flag = false;

BufferedOutputStream bufferedOutputStream = null;
BufferedInputStream bufferedInputStream = null;
try {
bufferedInputStream = new BufferedInputStream(file.getInputStream());
bufferedOutputStream = new BufferedOutputStream(new FileOutputStream(toLocalFilePath));
int index;
byte[] bytes = new byte[4096];
while ((index = bufferedInputStream.read(bytes)) != -1) {
bufferedOutputStream.write(bytes, 0, index);
bufferedOutputStream.flush();
}
flag = true;
} catch (IOException e) {
log.error("文件写入失败," + e.getMessage());
if (new File(toLocalFilePath).exists()) {
new File(toLocalFilePath).delete();
}
throw new Exception(e.getMessage());
} finally {
if (bufferedOutputStream != null) {
try {
bufferedOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (bufferedInputStream != null) {
try {
bufferedInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
System.gc();
}
shiyu6226
OP
  
@Stendan
感谢,目前看来好像只有分片上传可行了
Xhack   
try-with-resource
您需要登录后才可以回帖 登录 | 立即注册

返回顶部