apoc.export.cypher.all

此过程不建议在多线程中运行,因此并行运行时(Parallel runtime)不支持该过程。有关更多信息,请参阅 Cypher 手册 → 并行运行时

详细信息

语法

apoc.export.cypher.all([ file, config ]) :: (file, batches, source, format, nodes, relationships, properties, time, rows, batchSize, cypherStatements, nodeStatements, relationshipStatements, schemaStatements, cleanupStatements)

描述

将整个数据库(包含索引)以 Cypher 语句的形式导出到指定文件(默认格式:Cypher Shell)。

输入参数

名称

类型

描述

file

STRING

数据导出到的文件名。默认值为:``。

config

MAP

{ stream = false :: BOOLEAN, batchSize = 20000 :: INTEGER, bulkImport = false :: BOOLEAN, timeoutSeconds = 100 :: INTEGER, compression = 'None' :: STRING, charset = 'UTF_8' :: STRING, sampling = false :: BOOLEAN, samplingConfig :: MAP }。默认值为:{}

返回参数

名称

类型

描述

file

STRING

数据导出到的文件名。

batches

INTEGER(整数)

导出过程执行的批次数。

source

STRING

导出数据的摘要。

format

STRING

文件的导出格式。

节点

INTEGER(整数)

已导出节点的数量。

relationships

INTEGER(整数)

已导出关系的数量。

属性

INTEGER(整数)

已导出属性的数量。

time

INTEGER(整数)

导出所花费的时间。

rows

INTEGER(整数)

返回的行数。

batchSize

INTEGER(整数)

导出过程执行的批处理大小。

cypherStatements

ANY

已执行的 Cypher 语句。

nodeStatements

ANY

已执行的节点语句。

relationshipStatements

ANY

已执行的关系语句。

schemaStatements

ANY

已执行的模式(Schema)语句。

cleanupStatements

ANY

已执行的清理语句。

配置参数

该过程支持以下配置参数

配置参数
名称 类型 默认 描述

writeNodeProperties

布尔值 (BOOLEAN)

false

如果为 true,则同时导出属性。

stream

`BOOLEAN

false

将 JSON 直接流式传输给客户端的 data 字段。

format

STRING

cypher-shell

导出格式。支持以下值:

  • cypher-shell - 用于通过 Cypher Shell 导入

  • neo4j-shell - 用于通过 Neo4j Shell 导入

  • plain - 导出纯 Cypher,不含 begincommitawait 命令。用于通过 Neo4j Browser 导入

cypherFormat

STRING

create

Cypher 更新操作类型。支持以下值:

  • create - 仅使用 CREATE 子句

  • updateAll - 使用 MERGE 代替 CREATE

  • addStructure - 对节点使用 MATCH,对关系使用 MERGE

  • updateStructure - 对节点和关系均使用 MERGEMATCH

useOptimizations

MAP

{type: "UNWIND_BATCH", unwindBatchSize: 20}

用于生成 Cypher 语句的优化配置。type 支持以下值:

  • NONE - 使用 CREATE 语句导出文件

  • UNWIND_BATCH - 通过 UNWIND 方法批量导出实体,详情请参考 Michael Hunger 关于快速批量写入的文章。

  • UNWIND_BATCH_PARAMS - 与 UNWIND_BATCH 类似,但在适当的情况下也会使用参数

awaitForIndexes

INTEGER(整数)

300

当使用 format: "cypher-shell" 时,用于 db.awaitIndexes 的超时时间

ifNotExists

布尔值 (BOOLEAN)

false

如果为 true,则在约束和索引中添加 IF NOT EXISTS 关键字。

导出到文件

默认情况下,导出到文件系统是被禁用的。我们可以通过在 apoc.conf 中设置以下属性来启用它:

apoc.conf
apoc.export.file.enabled=true

有关访问 apoc.conf 的更多信息,请参阅配置选项章节。

如果我们尝试在未先设置此属性的情况下使用任何导出过程,我们将收到以下错误消息:

Failed to invoke procedure: Caused by: java.lang.RuntimeException: Export to files not enabled, please set apoc.export.file.enabled=true in your apoc.conf. Otherwise, if you are running in a cloud environment without filesystem access, use the {stream:true} config and null as a 'file' parameter to stream the export back to your client.

导出文件被写入 import 目录,该目录由 server.directories.import 属性定义。这意味着我们提供的任何文件路径都是相对于此目录的。如果我们尝试写入绝对路径(例如 /tmp/filename),我们将收到类似于以下内容的错误消息:

Failed to invoke procedure: Caused by: java.io.FileNotFoundException: /path/to/neo4j/import/tmp/fileName (No such file or directory)

我们可以通过在 apoc.conf 中设置以下属性来允许写入文件系统上的任何位置:

apoc.conf
apoc.import.file.use_neo4j_config=false

Neo4j 现在将能够写入文件系统上的任何位置,因此在设置此属性之前,请确保这是您的意图。

导出流

如果我们不想导出到文件,可以通过将文件名设置为 null 并提供 stream:true 配置,将结果流式传输回 data 列。

使用示例

本节中的示例基于以下示例图

CREATE (TheMatrix:Movie {title:'The Matrix', released:1999, tagline:'Welcome to the Real World'})
CREATE (Keanu:Person {name:'Keanu Reeves', born:1964})
CREATE (Carrie:Person {name:'Carrie-Anne Moss', born:1967})
CREATE (Laurence:Person {name:'Laurence Fishburne', born:1961})
CREATE (Hugo:Person {name:'Hugo Weaving', born:1960})
CREATE (LillyW:Person {name:'Lilly Wachowski', born:1967})
CREATE (LanaW:Person {name:'Lana Wachowski', born:1965})
CREATE (JoelS:Person {name:'Joel Silver', born:1952})
CREATE
(Keanu)-[:ACTED_IN {roles:['Neo']}]->(TheMatrix),
(Carrie)-[:ACTED_IN {roles:['Trinity']}]->(TheMatrix),
(Laurence)-[:ACTED_IN {roles:['Morpheus']}]->(TheMatrix),
(Hugo)-[:ACTED_IN {roles:['Agent Smith']}]->(TheMatrix),
(LillyW)-[:DIRECTED]->(TheMatrix),
(LanaW)-[:DIRECTED]->(TheMatrix),
(JoelS)-[:PRODUCED]->(TheMatrix);

下方的 Neo4j Browser 可视化显示了导入的图

play movies
图 1. 电影图谱可视化

导出为 Cypher Shell 格式

默认情况下,导出 Cypher 过程生成的 Cypher 语句采用的是 Cypher Shell 格式。

以下查询使用默认的 UNWIND_BATCH 优化方式,将整个数据库以默认的 cypher-shell 格式导出到 all.cypher

// default config populated for illustration
CALL apoc.export.cypher.all("all.cypher", {
    format: "cypher-shell",
    useOptimizations: {type: "UNWIND_BATCH", unwindBatchSize: 20}
})
YIELD file, batches, source, format, nodes, relationships, properties, time, rows, batchSize
RETURN file, batches, source, format, nodes, relationships, properties, time, rows, batchSize;
结果
file batches source format 节点 relationships 属性 time rows batchSize

"all.cypher"

1

"database: nodes(8), rels(7)"

"cypher"

8

7

21

10

15

20000

all.cypher 的内容如下所示(为便于阅读,已添加额外空行):

all.cypher
:begin
CREATE CONSTRAINT uniqueConstraint FOR (node:`UNIQUE IMPORT LABEL`) REQUIRE (node.`UNIQUE IMPORT ID`) IS UNIQUE;
:commit

:begin
UNWIND [{_id:0, properties:{tagline:"Welcome to the Real World", title:"The Matrix", released:1999}}] AS row
CREATE (n:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row._id}) SET n += row.properties SET n:Movie;

UNWIND [{_id:1, properties:{born:1964, name:"Keanu Reeves"}}, {_id:2, properties:{born:1967, name:"Carrie-Anne Moss"}}, {_id:3, properties:{born:1961, name:"Laurence Fishburne"}}, {_id:4, properties:{born:1960, name:"Hugo Weaving"}}, {_id:5, properties:{born:1967, name:"Lilly Wachowski"}}, {_id:6, properties:{born:1965, name:"Lana Wachowski"}}, {_id:7, properties:{born:1952, name:"Joel Silver"}}] AS row
CREATE (n:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row._id}) SET n += row.properties SET n:Person;
:commit

:begin
UNWIND [{start: {_id:1}, end: {_id:0}, properties:{roles:["Neo"]}}, {start: {_id:2}, end: {_id:0}, properties:{roles:["Trinity"]}}, {start: {_id:3}, end: {_id:0}, properties:{roles:["Morpheus"]}}, {start: {_id:4}, end: {_id:0}, properties:{roles:["Agent Smith"]}}] AS row
MATCH (start:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.start._id})
MATCH (end:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.end._id})
CREATE (start)-[r:ACTED_IN]->(end) SET r += row.properties;

UNWIND [{start: {_id:7}, end: {_id:0}, properties:{}}] AS row
MATCH (start:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.start._id})
MATCH (end:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.end._id})
CREATE (start)-[r:PRODUCED]->(end) SET r += row.properties;

UNWIND [{start: {_id:5}, end: {_id:0}, properties:{}}, {start: {_id:6}, end: {_id:0}, properties:{}}] AS row
MATCH (start:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.start._id})
MATCH (end:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.end._id})
CREATE (start)-[r:DIRECTED]->(end) SET r += row.properties;
:commit

:begin
MATCH (n:`UNIQUE IMPORT LABEL`)  WITH n LIMIT 20000 REMOVE n:`UNIQUE IMPORT LABEL` REMOVE n.`UNIQUE IMPORT ID`;
:commit

:begin
DROP CONSTRAINT uniqueConstraint;
:commit

该 Cypher 脚本执行 5 个事务,每个事务由 :begin:commit 命令包裹。事务执行以下操作:

  1. UNIQUE IMPORT LABEL 标签和 UNIQUE IMPORT ID 属性上创建唯一约束

  2. 导入 PersonMovie 节点

  3. 在这些节点之间创建 ACTED_INPRODUCEDDIRECTED 关系

  4. 从节点中移除 UNIQUE IMPORT LABEL 标签和 UNIQUE IMPORT ID 属性

  5. 删除 UNIQUE IMPORT LABEL 标签和 UNIQUE IMPORT ID 属性上的唯一约束

此脚本可以使用 Cypher Shell 命令行工具执行。

例如,我们可以通过运行以下命令将 all.cypher 的内容导入到 Neo4j Aura 数据库中:

cat all.cypher | ./bin/cypher-shell -a <bolt-url> -u neo4j -p <password> --format verbose

别忘了将 <bolt-url> 和 <password> 替换为相应的凭据。

如果我们针对一个空数据库运行此命令,将会看到以下输出:

0 rows available after 70 ms, consumed after another 0 ms
Added 1 constraints
0 rows available after 16 ms, consumed after another 0 ms
Added 2 nodes, Set 8 properties, Added 4 labels
0 rows available after 40 ms, consumed after another 0 ms
Added 14 nodes, Set 42 properties, Added 28 labels
0 rows available after 51 ms, consumed after another 0 ms
Created 8 relationships, Set 8 properties
0 rows available after 38 ms, consumed after another 0 ms
Created 2 relationships
0 rows available after 38 ms, consumed after another 0 ms
Created 4 relationships
0 rows available after 20 ms, consumed after another 0 ms
Set 16 properties, Removed 16 labels
0 rows available after 3 ms, consumed after another 0 ms
Removed 1 constraints
故障排除

如果您在尝试导入时遇到失败,可以添加 --debug 命令行参数,以查看最后执行并导致失败的语句。

此外,请检查 Neo4j 实例的内存配置,建议通过在 neo4j.conf 中设置 dbms.memory.heap.max_size=2G 将堆内存(HEAP)大小增加到 2–4GB

我们还可以通过在命令前添加 JAVA_OPTS=-Xmx4G bin/cypher-shell … 来为 cypher-shell 本身提供更多内存。

如果我们没有文件系统访问权限,或者出于其他原因不想写入文件,可以流式传输导出语句。

以下查询在 cypherStatements 列中流式返回整个数据库:

CALL apoc.export.cypher.all(null, {
    batchSize: 5,
    streamStatements: true,
    format: "cypher-shell",
    useOptimizations: {type: "UNWIND_BATCH", unwindBatchSize: 5}
})
YIELD nodes, relationships, properties, cypherStatements
RETURN nodes, relationships, properties, cypherStatements;
结果
节点 relationships 属性 cypherStatements

16

0

34

":begin CREATE CONSTRAINT uniqueConstraint FOR (node:`UNIQUE IMPORT LABEL`) REQUIRE (node.UNIQUE IMPORT ID) IS UNIQUE; :commit :begin UNWIND [{_id:0, properties:{tagline:\"Welcome to the Real World\", title:\"The Matrix\", released:1999}}, {_id:1, properties:{tagline:\"Welcome to the Real World\", title:\"The Matrix\", released:1999}}] AS row CREATE (n:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row._id}) SET n += row.properties SET n:Movie; UNWIND [{_id:35, properties:{born:1967, name:\"Carrie-Anne Moss\"}}, {_id:36, properties:{born:1961, name:\"Laurence Fishburne\"}}, {_id:37, properties:{born:1965, name:\"Lana Wachowski\"}}] AS row CREATE (n:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row._id}) SET n += row.properties SET n:Person; :commit :begin UNWIND [{_id:38, properties:{born:1964, name:\"Keanu Reeves\"}}, {_id:39, properties:{born:1952, name:\"Joel Silver\"}}, {_id:40, properties:{born:1960, name:\"Hugo Weaving\"}}, {_id:41, properties:{born:1967, name:\"Lilly Wachowski\"}}, {_id:42, properties:{born:1967, name:\"Carrie-Anne Moss\"}}] AS row CREATE (n:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row._id}) SET n += row.properties SET n:Person; :commit :begin UNWIND [{_id:43, properties:{born:1965, name:\"Lana Wachowski\"}}, {_id:50, properties:{born:1960, name:\"Hugo Weaving\"}}, {_id:51, properties:{born:1964, name:\"Keanu Reeves\"}}, {_id:57, properties:{born:1967, name:\"Lilly Wachowski\"}}, {_id:58, properties:{born:1961, name:\"Laurence Fishburne\"}}] AS row CREATE (n:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row._id}) SET n += row.properties SET n:Person; :commit :begin UNWIND [{_id:59, properties:{born:1952, name:\"Joel Silver\"}}] AS row CREATE (n:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row._id}) SET n += row.properties SET n:Person; :commit "

16

14

42

":begin UNWIND [{start: {_id:35}, end: {_id:0}, properties:{roles:[\"Trinity\"]}}, {start: {_id:36}, end: {_id:0}, properties:{roles:[\"Morpheus\"]}}, {start: {_id:50}, end: {_id:1}, properties:{roles:[\"Agent Smith\"]}}, {start: {_id:40}, end: {_id:0}, properties:{roles:[\"Agent Smith\"]}}, {start: {_id:51}, end: {_id:1}, properties:{roles:[\"Neo\"]}}] AS row MATCH (start:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.start._id}) MATCH (end:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.end._id}) CREATE (start)-[r:ACTED_IN]→(end) SET r += row.properties; :commit :begin UNWIND [{start: {_id:42}, end: {_id:1}, properties:{roles:[\"Trinity\"]}}, {start: {_id:38}, end: {_id:0}, properties:{roles:[\"Neo\"]}}, {start: {_id:58}, end: {_id:1}, properties:{roles:[\"Morpheus\"]}}] AS row MATCH (start:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.start._id}) MATCH (end:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.end._id}) CREATE (start)-[r:ACTED_IN]→(end) SET r += row.properties; UNWIND [{start: {_id:59}, end: {_id:1}, properties:{}}, {start: {_id:39}, end: {_id:0}, properties:{}}] AS row MATCH (start:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.start._id}) MATCH (end:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.end._id}) CREATE (start)-[r:PRODUCED]→(end) SET r += row.properties; :commit :begin UNWIND [{start: {_id:37}, end: {_id:0}, properties:{}}, {start: {_id:57}, end: {_id:0}, properties:{}}, {start: {_id:43}, end: {_id:1}, properties:{}}, {start: {_id:41}, end: {_id:1}, properties:{}}] AS row MATCH (start:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.start._id}) MATCH (end:`UNIQUE IMPORT LABEL`{UNIQUE IMPORT ID: row.end._id}) CREATE (start)-[r:DIRECTED]→(end) SET r += row.properties; :commit "

16

14

42

":begin MATCH (n:`UNIQUE IMPORT LABEL`) WITH n LIMIT 5 REMOVE n:`UNIQUE IMPORT LABEL` REMOVE n.UNIQUE IMPORT ID; :commit :begin MATCH (n:`UNIQUE IMPORT LABEL`) WITH n LIMIT 5 REMOVE n:`UNIQUE IMPORT LABEL` REMOVE n.UNIQUE IMPORT ID; :commit :begin MATCH (n:`UNIQUE IMPORT LABEL`) WITH n LIMIT 5 REMOVE n:`UNIQUE IMPORT LABEL` REMOVE n.UNIQUE IMPORT ID; :commit :begin MATCH (n:`UNIQUE IMPORT LABEL`) WITH n LIMIT 5 REMOVE n:`UNIQUE IMPORT LABEL` REMOVE n.UNIQUE IMPORT ID; :commit :begin DROP CONSTRAINT uniqueConstraint; :commit "

然后,我们可以将 cypherStatements 列的内容(不含双引号)复制/粘贴到 Cypher Shell 会话中,或者粘贴到本地文件中,并将该文件流式传输到 Cypher Shell 会话中。

导出为 Neo4j Browser 友好格式

Cypher 导出过程支持配置 format: "plain",这对于稍后使用 Neo4j Browser 导入非常有用。

以下查询将整个数据库导出到 all-plain.cypher

CALL apoc.export.cypher.all("all-plain.cypher", {
    format: "plain",
    useOptimizations: {type: "UNWIND_BATCH", unwindBatchSize: 20}
})
YIELD file, batches, source, format, nodes, relationships, properties, time, rows, batchSize
RETURN file, batches, source, format, nodes, relationships, properties, time, rows, batchSize;
结果
file batches source format 节点 relationships 属性 time rows batchSize

"all-plain.cypher"

1

"database: nodes(8), rels(7)"

"cypher"

8

7

21

9

15

20000

all-plain.cypher 的内容如下所示(为便于阅读,已添加额外空行):

all-plain.cypher
CREATE CONSTRAINT uniqueConstraint FOR (node:`UNIQUE IMPORT LABEL`) REQUIRE (node.`UNIQUE IMPORT ID`) IS UNIQUE;

UNWIND [{_id:0, properties:{tagline:"Welcome to the Real World", title:"The Matrix", released:1999}}] AS row
CREATE (n:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row._id}) SET n += row.properties SET n:Movie;

UNWIND [{_id:1, properties:{born:1964, name:"Keanu Reeves"}}, {_id:2, properties:{born:1967, name:"Carrie-Anne Moss"}}, {_id:3, properties:{born:1961, name:"Laurence Fishburne"}}, {_id:4, properties:{born:1960, name:"Hugo Weaving"}}, {_id:5, properties:{born:1967, name:"Lilly Wachowski"}}, {_id:6, properties:{born:1965, name:"Lana Wachowski"}}, {_id:7, properties:{born:1952, name:"Joel Silver"}}] AS row
CREATE (n:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row._id}) SET n += row.properties SET n:Person;

UNWIND [{start: {_id:1}, end: {_id:0}, properties:{roles:["Neo"]}}, {start: {_id:2}, end: {_id:0}, properties:{roles:["Trinity"]}}, {start: {_id:3}, end: {_id:0}, properties:{roles:["Morpheus"]}}, {start: {_id:4}, end: {_id:0}, properties:{roles:["Agent Smith"]}}] AS row
MATCH (start:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.start._id})
MATCH (end:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.end._id})
CREATE (start)-[r:ACTED_IN]->(end) SET r += row.properties;

UNWIND [{start: {_id:7}, end: {_id:0}, properties:{}}] AS row
MATCH (start:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.start._id})
MATCH (end:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.end._id})
CREATE (start)-[r:PRODUCED]->(end) SET r += row.properties;

UNWIND [{start: {_id:5}, end: {_id:0}, properties:{}}, {start: {_id:6}, end: {_id:0}, properties:{}}] AS row
MATCH (start:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.start._id})
MATCH (end:`UNIQUE IMPORT LABEL`{`UNIQUE IMPORT ID`: row.end._id})
CREATE (start)-[r:DIRECTED]->(end) SET r += row.properties;

MATCH (n:`UNIQUE IMPORT LABEL`)  WITH n LIMIT 20000 REMOVE n:`UNIQUE IMPORT LABEL` REMOVE n.`UNIQUE IMPORT ID`;

DROP CONSTRAINT uniqueConstraint;

接着,我们可以获取 all-plain.cypher 文件并将其拖动到 Neo4j Browser 窗口中。然后应该会看到以下提示:

export cypher plain drag
图 2. 将文件拖入 Neo4j Browser 时的提示

点击 Paste in editor(在编辑器中粘贴)后,文件内容将出现在查询编辑器中:

export cypher plain editor
图 3. 包含 all-plain.cypher 内容的 Neo4j Browser 查询编辑器

然后我们可以按下编辑器旁边的播放按钮,数据即可完成导入。