jt-pve-storage-purestorage

Disclaimer 免責聲明

Features 功能特性

Direct integration with Pure Storage FlashArray, leveraging native array capabilities through the REST API. 直接整合 Pure Storage FlashArray，透過 REST API 充分運用原生陣列功能。

Requirements 系統需求

Proxmox VE

Pure Storage FlashArray

• Purity//FA 6.x or later (REST API 1.16+ or 2.0+ supported) Purity//FA 6.x 或更新版本（支援 REST API 1.16+ 或 2.0+）
• REST API enabled (default on FlashArray) 已啟用 REST API（FlashArray 預設啟用）
• Network connectivity from Proxmox VE management network to the array management IP (HTTPS/443) Proxmox VE 管理網路可連線至陣列管理 IP（HTTPS/443）
• iSCSI or FC ports configured and reachable from Proxmox VE nodes 已設定 iSCSI 或 FC port，且 Proxmox VE 節點可連線
• User account with API token, or username + password with permissions for volume, host, snapshot, and pod operations 有 API token 或 帳號密碼的使用者，需有 volume、host、snapshot、pod 操作權限
• For ActiveCluster: a configured pod with the source array as a member 使用 ActiveCluster 時：已設定的 pod，且來源陣列為成員

Proxmox VE Node Dependencies Proxmox VE 節點相依套件

Installation 安裝

First-Time Installation 首次安裝

apt update

apt install -y open-iscsi multipath-tools sg3-utils psmisc \
    libwww-perl libjson-perl liburi-perl lsscsi

systemctl enable --now iscsid
systemctl enable --now multipathd

dpkg -i jt-pve-storage-purestorage_1.1.37-1_all.deb

Fix Broken State 修復安裝失敗狀態

If you ran dpkg -i before installing dependencies and got errors: 如果在安裝相依套件前就執行 dpkg -i 而出現錯誤：

apt update
apt --fix-broken install -y
dpkg -l | grep jt-pve-storage-purestorage

Cluster Installation 叢集安裝

Repeat the 4 steps above on EACH node. Install the plugin on ALL nodes first, then add the storage configuration (only once, on any node). 在每個節點上重複上述 4 個步驟。先在所有節點安裝 plugin，再新增儲存設定（只需在任一節點操作一次）。

Upgrade SOP 升級標準作業程序

When upgrading, follow these steps on every cluster node in sequence (one node at a time): 升級時，請依序在每個叢集節點上執行以下步驟（一次一個節點）：

# Backup multipath configs
cp /etc/multipath.conf /etc/multipath.conf.bak.$(date +%Y%m%d-%H%M%S)
cp /etc/multipath/conf.d/pure-storage.conf \
    /etc/multipath/conf.d/pure-storage.conf.bak.$(date +%Y%m%d-%H%M%S) 2>/dev/null
# Note current version
dpkg -l jt-pve-storage-purestorage | tail -1

dpkg -i jt-pve-storage-purestorage_1.1.37-1_all.deb

# If postinst warned about dangerous multipath.conf settings, edit:
nano /etc/multipath.conf
# Apply: no_path_retry queue -> 30, remove queue_if_no_path, dev_loss_tmo infinity -> 60
systemctl restart multipathd  # restart, NOT reload

# Verify
dpkg -l jt-pve-storage-purestorage | grep ii
pvesm status | grep purestorage
multipath -ll

Quick Start 快速入門

Add Storage (iSCSI, API token) 新增儲存（iSCSI、API token）

pvesm add purestorage pure1 \
    --pure-portal 192.168.1.100 \
    --pure-api-token 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' \
    --pure-protocol iscsi \
    --content images,rootdir \
    --shared 1

Add Storage (FC / Fibre Channel) 新增儲存（FC / Fibre Channel）

pvesm add purestorage pure-fc \
    --pure-portal 192.168.1.100 \
    --pure-api-token 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' \
    --pure-protocol fc \
    --content images \
    --shared 1

Add Storage (ActiveCluster Pod) 新增儲存（ActiveCluster Pod）

pvesm add purestorage pure-stretched \
    --pure-portal 192.168.1.100 \
    --pure-api-token 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' \
    --pure-protocol iscsi \
    --pure-pod prod-pod \
    --content images,rootdir \
    --shared 1

Verify 驗證

pvesm status
# Name      Type         Status  Total   Used  Available
# pure1     purestorage  active  ...     ...   ...

Configuration Reference 設定選項參考

Required Options 必要選項

Optional Options 可選選項

Proxmox VE Standard Storage Options Proxmox VE 標準儲存選項

These are standard Proxmox VE storage options that apply to all storage types, including this plugin. 這些是 Proxmox VE 標準儲存選項，適用於所有儲存類型，包含本外掛程式。

Capacity and Permission Isolation: Pod and Realm 容量與權限隔離：Pod 與 Realm

The plugin connects to the array via REST API and creates Volumes on demand. By default it uses the entire array's capacity and an array-wide API token. To bound what the plugin can touch -- both for capacity caps and for API blast-radius isolation -- Pure offers two mechanisms the plugin honours. 外掛透過 REST API 連接陣列，並依需求動態建立 Volume。預設情況下，外掛使用整台陣列的容量，且 API Token 為陣列層級權限。若要限制外掛可動用的範圍——無論是出於容量上限或 API 影響範圍隔離考量——Pure 提供兩種外掛能識別的機制。

Volume names always use the pve-<storeid>- prefix, so existing array Volumes that do NOT match this prefix are never read or written by the plugin even without any of the isolation options below. Volume 名稱永遠採用 pve-<storeid>- 前綴，因此既有陣列上不符合此前綴的 Volume，即使未啟用以下任何隔離選項，也不會被外掛讀取或寫入。

Option A -- Pod with quota (any Purity version) 方案 A——Pod 加配額（任何 Purity 版本）

A Pod is a namespace on the array. Set a quota_limit on the Pod and point the plugin at it; the plugin then reports and enforces the Pod's quota instead of the array total. Volume names get a pod:: prefix so they cannot collide with anything outside the Pod. Pod 是陣列上的命名空間。在 Pod 設定 quota_limit 並讓外掛指向此 Pod，外掛便會以 Pod 配額取代陣列總容量進行回報與管理。Volume 名稱會自動加上 pod:: 前綴，與 Pod 外的物件不會撞名。

On Pure: Storage > Pods > Create (e.g. pve-pod), then edit the Pod and set Quota Limit (e.g. 2T). Pure 端：Storage > Pods > Create（名稱例如 pve-pod），編輯該 Pod 並設定 Quota Limit（例如 2T）。

On Proxmox VE: Proxmox VE 端：

pvesm add purestorage pure1 \
    --pure-portal 192.168.1.100 \
    --pure-api-token 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' \
    --pure-protocol iscsi \
    --pure-pod pve-pod \
    --content images,rootdir \
    --shared 1

Single-array (local) Pods do not require an ActiveCluster license; ActiveCluster is only needed for stretched (cross-array) Pods. 單陣列 local Pod 不需要 ActiveCluster 授權；ActiveCluster 僅用於跨陣列的 stretched Pod。

Option B -- Realm with quota (Purity 6.6+) 方案 B——Realm 加配額（Purity 6.6+）

A Realm is a tenant-level isolation boundary on the array. Volumes, snapshots, hosts, and the API capacity view are all scoped to the realm. An API token created for a user inside the realm has no visibility into anything outside. Realm 是陣列上的租戶層級隔離邊界。Volume、Snapshot、Host、甚至 API 看到的容量視圖都被限定在 realm 內。在 realm 內建立的使用者所產生的 API Token，完全看不到 realm 之外的任何東西。

On Pure: Pure 端：

1. Settings > Access > Realms > Create -- name e.g. pve-realm, set Quota Limit (e.g. 2T) Settings > Access > Realms > Create——名稱例如 pve-realm，設定 Quota Limit（例如 2T）
2. Settings > Access > Users > Create -- username pve-plugin, realm pve-realm, role storage_admin Settings > Access > Users > Create——使用者 pve-plugin、Realm pve-realm、Role storage_admin
3. Open pve-plugin > API Tokens > Create 點 pve-plugin > API Tokens > Create

On Proxmox VE: just use the realm-scoped token. No plugin option to set -- the scoping is enforced by the token itself. Proxmox VE 端：直接使用此 realm-scoped Token，外掛沒有任何選項要設——範圍由 Token 本身強制執行。

pvesm add purestorage pure1 \
    --pure-portal 192.168.1.100 \
    --pure-api-token '<realm-scoped-token>' \
    --pure-protocol iscsi \
    --content images,rootdir \
    --shared 1

Choosing between them 兩者比較

Realm is the cleaner choice when available -- it isolates capacity AND permissions in a single mechanism. On older Purity, Pod is the only built-in mechanism the plugin recognises for capacity reporting; the Pod's quota_limit still gives you a hard capacity cap that the plugin honours, even though API tokens remain array-wide. 若版本支援，Realm 是比較乾淨的選擇——它在單一機制中同時隔離容量與權限。在較舊的 Purity 上，Pod 是唯一外掛能識別的容量機制；Pod 的 quota_limit 仍會給你硬上限，雖然 API Token 本身仍是陣列層級的權限。

Example storage.cfg (iSCSI, API token) 範例 storage.cfg (iSCSI、API token)

Example storage.cfg (FC SAN) 範例 storage.cfg (FC SAN)

Example storage.cfg (ActiveCluster Pod) 範例 storage.cfg (ActiveCluster Pod)

Architecture 系統架構

1:1 Architecture Model 1:1 架構模型

This plugin uses direct volume provisioning: every VM disk is allocated as its own dedicated Pure Storage volume, sized exactly to the requested capacity. There is no pre-allocated "big LUN" or storage pool that gets carved into smaller pieces on the host side, and no LVM-thin or FlexVol-style intermediate container. When Proxmox VE asks for a 50 GiB disk, the plugin calls volume_create on the array with size = 50 GiB. When Proxmox VE asks for a snapshot, the plugin calls snapshot_create on that volume. The semantics map 1:1, so Pure's data services (deduplication, compression, snapshot, ActiveCluster, async replication) all apply at the natural unit of one VM disk. 本外掛採 direct volume provisioning（直接 Volume 供應）：每一顆 VM 磁碟在 Pure FlashArray 上對應一個獨立的 Volume，大小即為當下要求的容量。陣列上沒有任何「預先建立的大 LUN」或儲存集區在主機端被切片，也沒有 LVM-thin 或 FlexVol 那種中間容器層。當 Proxmox VE 要求 50 GiB 磁碟時，外掛對陣列呼叫 volume_create 建立一顆 50 GiB 的 Volume；要求快照時，外掛對該 Volume 呼叫 snapshot_create。語意 1:1 對齊，Pure 的資料服務（重複資料刪除、壓縮、快照、ActiveCluster、非同步複寫）全部以「一顆 VM 磁碟」為自然單位作用。

Direct Volume Provisioning vs Pool-Based SAN 直接 Volume 供應 vs 儲存集區式 SAN

The pool-based model puts an LVM (or LVM-thin) layer between the array and Proxmox VE. Snapshots, clones, and replication happen at the LVM layer — the array sees a single giant LUN that is permanently "in use", and its data services cannot reason about individual VM disks. This plugin removes that intermediate layer entirely. 儲存集區式模型在陣列與 Proxmox VE 之間插入一層 LVM（或 LVM-thin）。快照、複製、複寫都發生在 LVM 層 — 陣列只看到一顆永遠「使用中」的大 LUN，無法針對單一 VM 磁碟做 dedup ratio、replication policy、snapshot retention。本外掛完全移除這層中介。

Layer Stack 層級堆疊

How a single Proxmox VE volname becomes a usable block device on the host: 一個 Proxmox VE volname 如何在主機端變成可用的區塊裝置：

What This Means in Practice 對使用者的意義

• Snapshots are array-native — volume_snapshot calls snapshot_create on Pure; rollback uses volume_overwrite. No LVM copy-on-write cost. 快照即陣列原生 — volume_snapshot 呼叫 Pure 的 snapshot_create；倒回使用 volume_overwrite。沒有 LVM copy-on-write 成本。
• Linked clones use array clone — instant; the new volume shares blocks with the source until written. Proxmox VE's "Full Clone" is a Proxmox VE design choice (block-by-block copy via qemu-img), not a plugin limit. 連結複製即陣列複製 — 瞬間完成；新 volume 與來源共享資料區塊直到被寫入。Proxmox VE 的「完整複製」是 Proxmox VE 設計選擇（透過 qemu-img 逐區塊複製），不是外掛限制。
• Online resize is one API call — volume_resize on the array, then per-path SCSI rescan + multipath_resize_map on the host. No filesystem-level resize on a parent pool. 線上擴充僅一次 API 呼叫 — 對陣列下 volume_resize，主機端再進行每路徑 SCSI rescan 與 multipath_resize_map。沒有上層儲存集區的檔案系統擴充步驟。
• Live migration is implicit — every volume is connected to every node's host (per-node mode), so the same multipath device exists on every cluster node simultaneously. Proxmox VE just changes which node has the device open; no storage migration step is needed. Live Migration 隱含完成 — 在 per-node 模式下，每個 volume 都連到每個節點的 host，相同的 multipath 裝置同時存在於叢集所有節點。Proxmox VE 只是切換哪個節點開啟它，無需儲存遷移步驟。
• Per-VM array data services — dedup ratio, replication policy, snapshot retention, QoS, and ActiveCluster Pod membership can all be configured per VM disk, because each disk is its own object on the array. 逐 VM 的陣列資料服務 — dedup 比、複寫政策、快照保留、QoS 與 ActiveCluster Pod 成員資格都可以針對單一 VM 磁碟設定，因為每顆磁碟在陣列上都是獨立物件。

Object Mapping 物件對應

Module Architecture 模組架構

VM Config Backup VM 設定備份

When you create a snapshot, the plugin automatically backs up the VM configuration file to Pure Storage. This means you can recover not only the disk data but also the VM settings as they were at that point in time. 建立快照時，Plugin 會自動將 VM 設定檔備份到 Pure Storage。這讓您不僅可以還原磁碟資料，也可以還原該時間點的 VM 設定。

How It Works 運作方式

• Automatic — the config is saved every time a snapshot is created. No extra setting required. 自動 — 每次建立快照都會儲存設定，不需任何額外設定。
• Per-snapshot — each snapshot gets its own independent config backup, so different points in time can have different configs. 獨立於每個快照 — 每個快照都有自己獨立的設定備份，不同時間點可以有不同的設定。
• Storage format — a 1MB ext4 volume named pve-{storage}-{vmid}-vmconf-{snapname} containing {vmid}.conf and a metadata.txt with timestamp and source info. 儲存格式 — 1MB 的 ext4 volume，命名為 pve-{storage}-{vmid}-vmconf-{snapname}，內含 {vmid}.conf 與標記時間戳與來源資訊的 metadata.txt。
• Hidden from UI — config volumes do not appear in the Proxmox VE disk listing. They are storage-side artifacts only. 不出現在 UI — 設定 volume 不會出現在 Proxmox VE 的磁碟清單中，純粹是儲存端的物件。

Automatic Cleanup 自動清理

• When a snapshot is deleted, its corresponding config volume is also deleted. 刪除快照時，對應的設定 volume 也會一併刪除。
• When the last disk of a VM is removed, all its config volumes are deleted along with it. 當 VM 的最後一顆磁碟被刪除時，該 VM 所有的設定 volume 也會被一併刪除。

Retrieving a Config Backup 取出設定備份

The package ships with a CLI tool pve-pure-config-get for retrieving config backups. 套件附帶 CLI 工具 pve-pure-config-get，用於取出設定備份。

pve-pure-config-get -s <storage> -v <vmid> [-n <snap>] [-o <output_dir>] [-l] [-r]

Examples 範例

# List available config backups for VM 100
pve-pure-config-get -s pure1 -v 100 -l

# Interactive retrieval (will prompt for snapshot selection)
pve-pure-config-get -s pure1 -v 100

# Retrieve a specific snapshot's config directly
pve-pure-config-get -s pure1 -v 100 -n snap1

# Retrieve to a custom directory
pve-pure-config-get -s pure1 -v 100 -n snap1 -o /root/configs

Output files are written as: 輸出檔案為：

/tmp/vm-{vmid}-{snapname}-{vmid}.conf      # VM config
/tmp/vm-{vmid}-{snapname}-metadata.txt     # Backup metadata

To restore the config back into Proxmox VE, copy the .conf file to /etc/pve/qemu-server/{vmid}.conf (or /etc/pve/lxc/{vmid}.conf for LXC). 要將設定還原到 Proxmox VE，將 .conf 檔複製到 /etc/pve/qemu-server/{vmid}.conf（LXC 則是 /etc/pve/lxc/{vmid}.conf）。

Disaster Recovery (-r mode) 災難復原（-r 模式）

The --restore flag enables full VM recovery from destroyed volumes that have not yet been eradicated by Pure Storage. This is the lifeline when a VM has been accidentally deleted: while Pure's destroyed-volume window is still open (default 24 hours), both the config backup and the disk volumes can be brought back. --restore 旗標啟用從 已刪除但尚未 eradicate 的 volume 完整復原 VM。當 VM 被誤刪、但 Pure 的 destroyed-volume 視窗（預設 24 小時）還沒關閉時，這就是救命稻草：設定備份與磁碟 volume 都可以救回。

• Searches both active and destroyed volumes; shows status as [active] or [DESTROYED]. 同時搜尋 active 與 destroyed 的 volume，狀態以 [active] 或 [DESTROYED] 顯示。
• Automatically recovers destroyed config and disk volumes from Pure Storage. 自動從 Pure Storage 救回已刪除的設定與磁碟 volume。
• Places the config file directly into the correct Proxmox VE location (/etc/pve/qemu-server/ or /etc/pve/lxc/). 直接將設定檔放置到 Proxmox VE 正確位置（/etc/pve/qemu-server/ 或 /etc/pve/lxc/）。
• Connects the recovered disk volumes to this host so the VM can start. 將救回的磁碟 volume 重新連到此主機，VM 即可啟動。
• Safety check: refuses to overwrite an existing VM config (use a different VM ID or delete the existing one first). 安全檢查：拒絕覆寫既有的 VM 設定（請改用其他 VM ID，或先刪除既有設定）。

# List available backups including destroyed volumes
pve-pure-config-get -s pure1 -v 100 -r -l

# Full VM restore from destroyed volumes
pve-pure-config-get -s pure1 -v 100 -n snap1 -r

Supported Features Matrix 功能支援表

Troubleshooting 疑難排解

Common issues and their solutions. For a comprehensive guide, see docs/TROUBLESHOOTING.md in the repository. 常見問題及解決方案。完整指南請參閱 repository 中的 docs/TROUBLESHOOTING_zh-TW.md。

Quick Diagnostic Commands 快速診斷指令

# Check storage status
pvesm status

# Check Proxmox VE daemon logs
journalctl -xeu pvedaemon --since "10 minutes ago"

# Check iSCSI sessions
iscsiadm -m session

# Check multipath devices
multipathd show maps

# Inspect WWID tracking state
cat /var/lib/pve-storage-purestorage/*-wwids.json

# Check Pure API connectivity
curl -k -H "api-token: YOUR-TOKEN" https://PURE_IP/api/2.21/arrays

# Rescan iSCSI sessions
iscsiadm -m session --rescan

# Reload multipath
multipathd reconfigure
multipath -v2

# Verify host connection on the array
# Pure UI: Storage > Hosts > pve-{cluster}-{node} > Connected Volumes

# 1. Disable queueing
multipathd disablequeueing map <wwid>
dmsetup message <wwid> 0 fail_if_no_path

# 2. Flush the specific device
multipath -f <wwid>

# 3. If step 2 fails, force remove
dmsetup remove --force --retry <wwid>

# Deactivate the guest VG on the host
vgchange -an <guest-vg-name>

# Long-term fix: add global_filter to /etc/lvm/lvm.conf
# global_filter = [ "r|/dev/mapper/3624a9370.*|", "a|.*|" ]

Changelog 變更紀錄

Version history. For full details see CHANGELOG.md. 版本紀錄。完整內容請參閱 CHANGELOG_zh-TW.md。

Acknowledgments 致謝

Special thanks to: 特別感謝：

• Pure Storage — for excellent storage technology and comprehensive REST API documentation that made dual 1.x/2.x support straightforward. Pure Storage 原廠 — 提供優秀的儲存技術與完善的 REST API，讓 1.x/2.x 雙版本支援得以順利實作。
• MetaAge (邁達特) — for providing test equipment and environment for development and testing. MetaAge 邁達特（代理商） — 協助提供測試設備與環境進行開發測試。
• Proxmox VE community — for the well-documented Storage Plugin API. Proxmox VE 社群 — 提供文件完善的 Storage Plugin API。

References 參考資料

• Proxmox Storage Plugin Development
• Pure Storage Support & Documentation
• Pure Storage Blog