急求请教 neo4j-admin import时报错,修改heap_size没有用
发布于 5 年前 作者 zxzcherish 2108 次浏览 来自 问答

请教各路神仙!急求支招

在cmd中使用neo4j-admin import 导入4.2G 的csv数据的时候,得到heap size may be too small 的提示,并且导入过程会停在40%-45%的中间,不再向下运行。 看了 http://neo4j.com.cn/topic/5acdac2f51bad0a10b198bb9 和其他的一些问答贴,使用 set HEAP_SIZE=500m、并且在neo4j.conf修改了dbms.memory.heap.initial_size和dbms.memory.heap.max_size修改之后, max heap memory的值至468.5MB; 但是只能导入1.4G的数据,再多也会卡住; (如果只修改dbms.memory.heap.initial_size和dbms.memory.heap.max_size 对heap memory没有改变)

如果set HEAP_SIZE = 1.3G,会报错: Invalid maximum heap size: -Xmx1.3G Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit.

改成set HEAP_SIZE = 1500m就可以了

后来又看了别的帖子说应该去改java -Xms -Xmx,我运行了几次 java -Xms512m -Xmx800m, java -Xms1024m -Xmx2048m, java -Xms2048m -Xmx2048m,总之都没有什么变化

现在导入数据不进去很着急,如果有明白这个原因的大神,请指点一下我应该怎么改~ 感谢!! 是windows系统,16G ram

以下是报错信息:(在设置HEAP_SIZE=1500m时)

Available resources:

Available resources: Total machine memory: 15.75 GB Free machine memory: 8.26 GB Max heap memory : 1.40 GB Processors: 8 Configured max memory: 12.92 GB High-IO: false

Import starting 2020-02-12 13:27:57.892+0800 Estimated number of nodes: 19.67 M Estimated number of node properties: 66.40 M Estimated number of relationships: 31.86 M Estimated number of relationship properties: 53.42 M Estimated disk space usage: 5.43 GB Estimated required memory usage: 1.24 GB

InteractiveReporterInteractions command list (end with ENTER): c: Print more detailed information about current stage i: Print more detailed information

(1/4) Node import 2020-02-12 13:27:57.919+0800 Estimated number of nodes: 19.67 M Estimated disk space usage: 2.37 GB Estimated required memory usage: 1.24 GB … … … … … 5% ?2s 732ms … … … … … 10% ?0ms … … … … … 15% ?0ms … … … … … 20% ?0ms … … … … … 25% ?15ms … … … … … 30% ?0ms … … … … … 35% ?0ms … … … … … 40% ?0ms … … … … … 45% ?0ms … … … … … 50% ?0ms … … … … … 55% ?16ms … … … … … 60% ?0ms … … … … … 65% ?0ms … … … … … 70% ?0ms … … … … … 75% ?15ms … … … … … 80% ?0ms … … … … … 85% ?0ms … … … … … 90% ?0ms … … … … … 95% ?0ms … … … … … 100% ?16ms

IMPORT FAILED in 5s 361ms. Data statistics is not available. Peak memory usage: 0.00 B WARNING Import failed. The store files in C:\Users\lzhang55.Neo4jDesktop\neo4jDatabases\database-51c75ac1-01de-49df-93bf-48cb2fcecbe5\installation-3.5.12\data\databases\graph.db are left as they are, although they are likely in an unusable state. Starting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually Exception in thread “main” org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 531960128 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320884736 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:400) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:420) at org.neo4j.io.mem.GrabAllocator$Grab.<init>(GrabAllocator.java:110) at org.neo4j.io.mem.GrabAllocator$Grabs.allocateAligned(GrabAllocator.java:263) at org.neo4j.io.mem.GrabAllocator.allocateAligned(GrabAllocator.java:83) at org.neo4j.io.pagecache.impl.muninn.PageList.initBuffer(PageList.java:299) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pageFault(MuninnPageCursor.java:391) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.uncommonPin(MuninnPageCursor.java:341) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pin(MuninnPageCursor.java:309) at org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor.next(MuninnWritePageCursor.java:96) at org.neo4j.kernel.impl.store.CommonAbstractStore.updateRecord(CommonAbstractStore.java:1120) at org.neo4j.kernel.impl.store.PropertyStore.updateRecord(PropertyStore.java:206) at org.neo4j.unsafe.impl.batchimport.EntityImporter.createAndWritePropertyChain(EntityImporter.java:152) at org.neo4j.unsafe.impl.batchimport.NodeImporter.endOfEntity(NodeImporter.java:144) at org.neo4j.unsafe.impl.batchimport.input.InputEntityVisitor$Delegate.endOfEntity(InputEntityVisitor.java:242) at org.neo4j.unsafe.impl.batchimport.input.InputEntityDecorators$AdditiveLabelsDecorator.endOfEntity(InputEntityDecorators.java:118) at org.neo4j.unsafe.impl.batchimport.input.csv.CsvInputParser.next(CsvInputParser.java:202) at org.neo4j.unsafe.impl.batchimport.input.csv.LazyCsvInputChunk.next(LazyCsvInputChunk.java:96) at org.neo4j.unsafe.impl.batchimport.input.csv.CsvInputChunkProxy.next(CsvInputChunkProxy.java:75) at org.neo4j.unsafe.impl.batchimport.ExhaustingEntityImporterRunnable.run(ExhaustingEntityImporterRunnable.java:57) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:122) Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 531960128 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError … 26 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 531960128 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError … 26 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 531960128 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError … 26 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 531960128 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:400) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:420) at org.neo4j.io.mem.GrabAllocator$Grab.<init>(GrabAllocator.java:110) at org.neo4j.io.mem.GrabAllocator$Grabs.allocateAligned(GrabAllocator.java:263) at org.neo4j.io.mem.GrabAllocator.allocateAligned(GrabAllocator.java:83) at org.neo4j.io.pagecache.impl.muninn.PageList.initBuffer(PageList.java:299) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pageFault(MuninnPageCursor.java:391) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.uncommonPin(MuninnPageCursor.java:341) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pin(MuninnPageCursor.java:309) at org.neo4j.io.pagecache.impl.muninn.MuninnWritePageCursor.next(MuninnWritePageCursor.java:96) at org.neo4j.kernel.impl.store.CommonAbstractStore.updateRecord(CommonAbstractStore.java:1120) at org.neo4j.kernel.impl.store.PropertyStore.updateRecord(PropertyStore.java:206) at org.neo4j.unsafe.impl.batchimport.EntityImporter.createAndWritePropertyChain(EntityImporter.java:163) … 13 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 537203008 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError … 26 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 537203008 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError … 26 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Suppressed: org.neo4j.unsafe.impl.internal.dragons.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 537203008 bytes have already been successfully allocated. The system currently has 16915566592 bytes total physical memory, 1934397440 bytes committed virtual memory, and 8320892928 bytes free physical memory. Relevant system properties: “java.vm.name” = “Java HotSpot™ Client VM”, “java.vm.vendor” = “Oracle Corporation”, “os.arch” = “x86”. The allocation was refused by the operating system: java.lang.OutOfMemoryError … 26 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more Caused by: java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at org.neo4j.unsafe.impl.internal.dragons.UnsafeUtil.allocateMemory(UnsafeUtil.java:396) … 25 more

1 回复

问题解决了/或者说没办法解决,是电脑的问题,16G太小了

回到顶部