graph-format.mdx 9.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244
  1. ---
  2. title: "Graph format"
  3. description: "Reference for the GraphNode and GraphEdge types used to represent entities and relationships in the Flowsint graph. Understanding these structures is essential for working with the frontend visualization and the graph database layer."
  4. category: "Developers"
  5. order: 11
  6. author: "Flowsint Team"
  7. tags: ["tutorial", "developers", "graph", "nodes", "edges"]
  8. version: "1.2.8"
  9. last_updated_at: "2026-05-15"
  10. ---
  11. ## Overview
  12. Flowsint's graph visualization is built on two core data structures: **GraphNode** (representing entities) and **GraphEdge** (representing relationships between entities). These types are defined in `flowsint-app/src/types/graph.ts` and are used throughout the frontend for rendering, interaction, and data management.
  13. Understanding these structures is important when:
  14. - Building enrichers that create nodes and relationships
  15. - Working on the frontend visualization
  16. - Debugging graph rendering issues
  17. - Extending the graph with new visual features
  18. ## GraphNode
  19. A `GraphNode` represents a single entity in the graph (e.g., a domain, an IP address, a person). Every node created by an enricher's `create_node()` method ends up as a `GraphNode` in the frontend.
  20. ```typescript
  21. type GraphNode = {
  22. id: string
  23. nodeType: string
  24. nodeLabel: string
  25. nodeProperties: NodeProperties
  26. nodeSize: number
  27. nodeColor: string | null
  28. nodeIcon: keyof typeof LucideIcons | null
  29. nodeImage: string | null
  30. nodeFlag: flagColor | null
  31. nodeShape: NodeShape | null
  32. nodeMetadata: NodeMetadata
  33. x: number
  34. y: number
  35. val?: number
  36. neighbors?: any[]
  37. links?: any[]
  38. }
  39. ```
  40. ### Field reference
  41. | Field | Type | Description |
  42. |-------|------|-------------|
  43. | `id` | `string` | Unique identifier for the node. Derived from the type's primary field value. |
  44. | `nodeType` | `string` | The entity type (e.g., `"Domain"`, `"Ip"`, `"Email"`). Maps to the Pydantic type class name. |
  45. | `nodeLabel` | `string` | Human-readable label displayed on the node. Set by the type's `compute_label()` method. |
  46. | `nodeProperties` | `NodeProperties` | All properties of the entity as key-value pairs. Contains the serialized fields from the Pydantic type. |
  47. | `nodeSize` | `number` | Visual size of the node in the graph. |
  48. | `nodeColor` | `string \| null` | Custom color override for the node (CSS color string), or `null` for default. |
  49. | `nodeIcon` | `keyof LucideIcons \| null` | Icon to display on the node, from the [Lucide icon set](https://lucide.dev/icons/), or `null` for default. |
  50. | `nodeImage` | `string \| null` | URL to an image to display on the node (e.g., a profile picture), or `null`. |
  51. | `nodeFlag` | `flagColor \| null` | Color flag for visual tagging (see Flag colors below), or `null`. |
  52. | `nodeShape` | `NodeShape \| null` | Shape of the node (see Node shapes below), or `null` for default circle. |
  53. | `nodeMetadata` | `NodeMetadata` | Additional metadata as key-value pairs. Used for internal tracking and extended features. |
  54. | `x` | `number` | X coordinate position in the graph canvas. |
  55. | `y` | `number` | Y coordinate position in the graph canvas. |
  56. | `val` | `number` (optional) | Value used by the force-graph engine for sizing calculations. |
  57. | `neighbors` | `any[]` (optional) | Adjacent nodes, populated by the graph engine at runtime. |
  58. | `links` | `any[]` (optional) | Connected edges, populated by the graph engine at runtime. |
  59. ### NodeProperties
  60. ```typescript
  61. type NodeProperties = {
  62. [key: string]: any
  63. }
  64. ```
  65. A flexible key-value object containing all the entity's properties. When a Pydantic type is serialized into a node, its fields become entries in `nodeProperties`. For example, a `Domain` node would have `nodeProperties.domain`, `nodeProperties.root`, etc.
  66. ### NodeMetadata
  67. ```typescript
  68. type NodeMetadata = {
  69. [key: string]: any
  70. }
  71. ```
  72. Similar to `nodeProperties` but used for system-level metadata rather than entity data. This can include information like creation timestamps, source enricher, or other tracking data.
  73. ### Node shapes
  74. Nodes can be rendered in four shapes:
  75. ```typescript
  76. type NodeShape = 'circle' | 'square' | 'hexagon' | 'triangle'
  77. ```
  78. | Shape | Use case |
  79. |-------|----------|
  80. | `circle` | Default shape for most entity types |
  81. | `square` | Often used for infrastructure entities |
  82. | `hexagon` | Used for grouped or aggregate entities |
  83. | `triangle` | Used for alert or warning-related entities |
  84. ### Flag colors
  85. Flags provide a visual tagging system for nodes. Users can flag nodes to highlight them during an investigation.
  86. ```typescript
  87. type flagColor = 'red' | 'orange' | 'blue' | 'green' | 'yellow'
  88. ```
  89. Each flag color maps to specific Tailwind CSS classes for consistent styling:
  90. | Flag | Style |
  91. |------|-------|
  92. | `red` | `text-red-400 fill-red-200` |
  93. | `orange` | `text-orange-400 fill-orange-200` |
  94. | `blue` | `text-blue-400 fill-blue-200` |
  95. | `green` | `text-green-400 fill-green-200` |
  96. | `yellow` | `text-yellow-400 fill-yellow-200` |
  97. ## GraphEdge
  98. A `GraphEdge` represents a relationship between two nodes (e.g., "RESOLVES_TO", "HAS_SUBDOMAIN", "FOUND_IN_BREACH"). Every relationship created by an enricher's `create_relationship()` method ends up as a `GraphEdge` in the frontend.
  99. ```typescript
  100. type GraphEdge = {
  101. source: GraphNode['id']
  102. target: GraphNode['id']
  103. date?: string
  104. id: string
  105. label: string
  106. caption?: string
  107. type?: string
  108. weight?: number
  109. confidence_level?: number | string
  110. }
  111. ```
  112. ### Field reference
  113. | Field | Type | Description |
  114. |-------|------|-------------|
  115. | `source` | `string` | ID of the source node (the "from" node in the relationship). |
  116. | `target` | `string` | ID of the target node (the "to" node in the relationship). |
  117. | `id` | `string` | Unique identifier for this edge. |
  118. | `label` | `string` | Display label for the edge (e.g., `"RESOLVES_TO"`, `"HAS_EMAIL"`). This is the relationship type string passed to `create_relationship()`. |
  119. | `date` | `string` (optional) | Timestamp associated with the relationship (ISO 8601 format). |
  120. | `caption` | `string` (optional) | Additional descriptive text displayed on or near the edge. |
  121. | `type` | `string` (optional) | Relationship classification type for filtering or styling. |
  122. | `weight` | `number` (optional) | Numerical weight of the relationship, can be used for visual thickness or importance ranking. |
  123. | `confidence_level` | `number \| string` (optional) | Confidence score for the relationship, indicating how reliable the connection is. |
  124. ## How enrichers map to the graph
  125. When you write an enricher and call graph methods in `postprocess()`, here's how the data flows:
  126. ### Creating nodes
  127. ```python
  128. # In your enricher's postprocess method
  129. self.create_node(domain) # domain is a Pydantic Domain object
  130. ```
  131. The graph service automatically:
  132. 1. Extracts the **type name** from the Pydantic class (e.g., `"Domain"`) -> `nodeType`
  133. 2. Reads the **primary field** value (e.g., `domain.domain`) -> used for `id`
  134. 3. Reads the **nodeLabel** field (set by `compute_label()`) -> `nodeLabel`
  135. 4. Serializes all fields into `nodeProperties`
  136. 5. Sets defaults for visual properties (`nodeSize`, `nodeColor`, etc.)
  137. ### Creating relationships
  138. ```python
  139. # In your enricher's postprocess method
  140. self.create_relationship(domain, ip, "RESOLVES_TO")
  141. ```
  142. The graph service automatically:
  143. 1. Identifies the **source node** from `domain`'s type and primary field -> `source`
  144. 2. Identifies the **target node** from `ip`'s type and primary field -> `target`
  145. 3. Uses the relationship label `"RESOLVES_TO"` -> `label`
  146. 4. Generates a unique `id` for the edge
  147. ### Relationship naming conventions
  148. Relationship labels follow these conventions:
  149. - Use **UPPER_SNAKE_CASE** (e.g., `"RESOLVES_TO"`, `"HAS_SUBDOMAIN"`)
  150. - Use **verb phrases** that describe the relationship direction (source -> target)
  151. - The default relationship label is `"IS_RELATED_TO"` if none is specified
  152. - Common patterns:
  153. - `HAS_*` for ownership/containment (e.g., `"HAS_EMAIL"`, `"HAS_SUBDOMAIN"`)
  154. - `RESOLVES_TO` for DNS-type lookups
  155. - `FOUND_IN_*` for breach/leak associations
  156. - `REGISTERED_BY` for WHOIS data
  157. ## Graph settings
  158. The graph visualization is configurable through settings types:
  159. ### ForceGraphSetting
  160. Controls the physics simulation of the force-directed graph layout:
  161. ```typescript
  162. type ForceGraphSetting = {
  163. value: any
  164. min?: number
  165. max?: number
  166. step?: number
  167. type?: string
  168. description?: string
  169. }
  170. ```
  171. ### ExtendedSetting
  172. General-purpose settings with richer type information:
  173. ```typescript
  174. type ExtendedSetting = {
  175. value: any
  176. type: string
  177. min?: number
  178. max?: number
  179. step?: number
  180. options?: { value: string; label: string }[]
  181. description?: string
  182. }
  183. ```
  184. These settings allow users to customize graph behavior like node repulsion, link distance, simulation speed, and visual preferences.
  185. ## Reserved properties
  186. When creating nodes, certain property names are reserved by the system and should not be used as field names in your custom types:
  187. - `id` - Node identifier
  188. - `x`, `y` - Position coordinates
  189. - `nodeLabel` - Display label (set by `compute_label()`)
  190. - `label` - Legacy label field
  191. - `nodeType`, `type` - Entity type identifiers
  192. - `nodeImage`, `nodeIcon`, `nodeColor`, `nodeSize` - Visual properties
  193. - `created_at` - Creation timestamp
  194. - `sketch_id` - Investigation sketch association