jt-pve-storage-synology

Which Proxmox VE operations work支援的 Proxmox VE 操作

Every row was driven on real hardware, from the web interface and from the command line, for virtual machines and for containers alike. The rows that are not a yes say why.每一列都在實機上、從網頁介面和指令列各跑過一次，虛擬機和容器都跑。不是「可以」的那幾列，都寫出原因。

Measured on hardware, not just called through an API實機驗證過，不只是 API 呼叫得通

Two models and three DSM versions: a DS918+ on 7.1.1 and 7.4.1, a DS925+ on 7.3.2. Both were driven from adding the storage through to removing it, leaving nothing behind on either side. Resize, clone, backup, migration and two-portal multipath were run on the DS918+ only. 兩台機型、三個 DSM 版本：DS918+ 的 7.1.1 與 7.4.1，DS925+ 的 7.3.2。兩台都從加入 storage 跑到移除，兩邊都沒有留下東西。擴充、複製、備份、遷移與雙 portal 多重路徑只在 DS918+ 上跑過。

Nine protocol findings changed a design decision each, and one rule the kernel's own source would lead you to get wrong: What the protocol turned out to be. 九個協定上的發現各自改變了一個設計決定，還有一條照著核心原始碼讀會讀錯的規則：協定實際上長什麼樣。

The release checklist發版前的操作清單

Every row was run on a production five-node cluster, and B through D once from the web interface and once from the CLI. The two run in different environments, so the same operation is not guaranteed to behave the same way, and that has caught five defects. What each row exercises: the checklist. 每一列都在一個五節點的正式叢集上跑過，B 到 D 從網頁介面和指令列各跑一次。網頁介面和指令列的執行環境不一樣，同一個操作不保證有同樣結果，而這件事抓到過五個缺陷。每一列走到什麼：操作清單。

What is verified, and what is not目前的驗證狀態

The same list in both directions, so neither half is easier to find than the other. 兩個方向都列出來，哪一半都不會比另一半難找。

Item by item, with what was measured and what was not: the test record. 逐項的量測結果，以及哪些還沒有量測過，都在驗證紀錄裡。

It is public because the discovery tool is useful on its own to anyone who wants to know what their DSM's SAN API offers —and because the record of what is and is not known about that API is worth reading before trusting any plugin built on it, including this one. 會公開，是因為探索工具本身就對想知道自己的 DSM 的 SAN API 提供什麼的人有用；也因為那份「哪些知道、哪些不知道」的紀錄，值得在信任任何建立在它之上的 plugin 之前先讀，包括這一個。

Requirements系統需求

The DSM accountDSM 帳號

Create a dedicated account for the plugin. It has to be in administrators, and everything else can be taken away. 為這個 plugin 建立一個專用帳號。它必須在 administrators 群組裡，而其他一切都可以拿掉。

# DSM → Control Panel → User & Group → Create
Name              pve-storage        # this is what --syno-username gets
Groups            administrators     # REQUIRED — DSM has nothing smaller, see below
Shared folders    No access          # to every one of them
Home folder       not created
Applications      Deny all except DSM   # the API logs in through DSM itself
Quota             none
Speed limit       none
2FA               optional           # the plugin can carry the device token

# then, still in Control Panel
Security → Firewall   allow the DSM port only from your PVE node addresses
Security → Account    leave Auto Block on — the plugin is written not to trip it

Never the built-in admin. And one thing DSM will not let you narrow, worth knowing before you plan the NAS's network: there is no "manage LUNs" privilege —no SAN operator role, and SAN Manager is not in the per-user Applications list— so the account must be an administrator. Everything else can still be taken away, and should be: every shared folder, every application but DSM, every address that is not a PVE node. A leaked credential then cannot read your files over SMB. 絕對不要用內建的 admin。而且在規劃 NAS 的網路之前，有一件 DSM 不讓你收窄的事最好先知道：沒有「管理 LUN」這種權限（沒有 SAN 操作員角色，應用程式權限清單裡也沒有 SAN Manager）所以帳號必須是管理員。其他一切仍然拿得掉，也應該拿掉：每一個共用資料夾、除 DSM 以外的每一個應用程式、每一個不屬於 PVE 節點的位址。這樣一組洩漏的憑證就翻不了你的檔案。

The account, its privileges, Auto Block, 2FA and TLS帳號、權限、自動封鎖、2FA 與 TLS

High availability and dual controllers高可用性與雙控制器

Synology has two arrangements that both get called "HA", and they are different problems. The plugin handles both, and neither has been verified on hardware yet. Synology 有兩種都被叫做「HA」的架構，而它們是兩個不同的問題。兩種架構 plugin 都已有對應處理，但目前皆尚未完成實機驗證。

syno-portal takes a list, tried in order and rotated on failure. The rotation happens inside the login and the request URL is built after it —a related project shipped a bug where the URL was built first, so every retry went on travelling to the address that had just been found dead. syno-portal 收一份清單，依序嘗試、失敗就輪替。輪替發生在登入裡面，而請求的 URL 是在輪替之後才組，所以重試會送往下一個位址，不會繼續送往剛剛才失敗的那一個。

On a UC chassis the second address need not be configured: SYNO.Core.Network.Interface accepts relay_node=node0 and node1 to enumerate the peer controller. On a single-controller NAS both answer with the same interfaces, so the mechanism is harmless where it is not needed —and on UC models a target's network_portals also carries a controller_id, which a single-controller NAS omits entirely. UC 機箱的第二個位址不必手動設定：SYNO.Core.Network.Interface 接受 relay_node=node0 與 node1 來列舉對側控制器。在單控制器的 NAS 上兩者回傳相同的介面，所以這個機制在不需要它的地方是無害的，而 UC 機型上 target 的 network_portals 還會帶 controller_id，單控制器的 NAS 完全不回傳這個欄位。

If you run either, one answer is worth more than any other report: after a failover, does SYNO.Core.ISCSI.Node still return the same uuid? The plugin uses that uuid as its answer to "which storage server is this", rather than the management address, because an address can be re-pointed at a different NAS. If the uuid changes across a failover, one NAS is read as two, and the approach itself has to change. 如果你手上有這兩種機型之一，最值得回報的就是一件事：故障切換之後，SYNO.Core.ISCSI.Node 回傳的 uuid 還是同一個嗎？plugin 拿這個 uuid 來認「這台儲存伺服器是誰」，而不是拿管理位址，因為位址可以被改指到另一台 NAS。如果 uuid 在切換之後會變，同一台 NAS 前後就會被認成兩台，那這個做法本身就得換掉。

Installing安裝

On every node in the cluster. Nothing needs restarting afterwards. 叢集中的每個節點都要裝。裝完不需要重啟任何東西。

# open-iscsi and multipath-tools are the two PVE does not install for you
apt update
apt install -y open-iscsi multipath-tools

cd /tmp
# no version in the filename: this URL always gives you the newest release
wget -O jt-pve-storage-synology_all.deb \
  https://github.com/jasoncheng7115/jt-pve-storage-synology/releases/latest/download/jt-pve-storage-synology_all.deb
apt install -y ./jt-pve-storage-synology_all.deb

# check what you got
dpkg -l jt-pve-storage-synology | awk '/^ii/{print $3}'

The filename carries no version on purpose. releases/latest/download/… always resolves to the newest release, so this command line stays correct after every release. If you want a specific version instead, the release page also carries the versioned file, jt-pve-storage-synology_<version>-1_all.deb. 檔名刻意不帶版本號。這個網址一律指向最新的一版，所以每次發布之後，同一段指令都還是對的。想裝特定版本的話，發布頁上也有帶版本號的檔案 jt-pve-storage-synology_<版本>-1_all.deb。

Every node in the cluster, including the one you browse from, on the same version. A node without the plugin makes the storage invisible in the web interface rather than reporting an error. The first activation runs multipathd reconfigure, which is node-wide —measured not to disturb another vendor's maps, but give a production node a window. 叢集中的每個節點都要裝，包含你正在瀏覽的那一台，版本要一致。沒有裝的節點會讓這個 storage 在網頁介面上看不到，而不是回報錯誤。第一次啟用會執行 multipathd reconfigure，那是節點層級的命令。實測不會動到別的廠商的 map，但正式節點請排個時段。

Where each of those comes from, and what to do if you installed with dpkg -i because an earlier version of this page said to: Notes for the curious. 上面這幾點各自的來由，還有如果你照本文件早期的版本用 dpkg -i 裝過，該怎麼處理，都寫在給想深入的人。

Upgrading更新

The same commands, plus one. apt install on a newer .deb upgrades in place and nothing needs restarting. 指令和安裝一樣，多一行。apt install 遇到比較新的 .deb 就是就地升級，不需要重啟任何東西。

cd /tmp
# the line that matters, see above
rm -f jt-pve-storage-synology_all.deb*
wget -O jt-pve-storage-synology_all.deb \
  https://github.com/jasoncheng7115/jt-pve-storage-synology/releases/latest/download/jt-pve-storage-synology_all.deb
apt install -y ./jt-pve-storage-synology_all.deb

dpkg -l jt-pve-storage-synology | awk '/^ii/{print $3}'

Do every node before you trust the result —a storage operation runs where the guest is, so a half-upgraded cluster behaves differently depending on which node a VM happens to be on. Running guests keep their devices: the package replaces Perl modules and does not touch iSCSI sessions, multipath maps or the drop-in. 每個節點都做完，才能相信結果：storage 操作是在 guest 所在的節點上跑的，所以升級到一半的叢集，行為會隨著 VM 剛好在哪一台而不同。執行中的 guest 會保有它們的裝置：這個套件換掉的是 Perl 模組，不會動到 iSCSI 工作階段、multipath 對應，或那個 drop-in。

The reasoning behind each line —Notes for the curious. 每一行背後的理由：給想深入的人。

Adding a Synology SAN storage in Proxmox VE在 Proxmox VE 新增 Synology SAN Storage

Run this once, on one node —unlike the install, which is every node in the cluster. The storage is shared by construction, so the others pick it up. 在其中一個節點上執行一次就好，和安裝相反，安裝是叢集中每個節點都要。這個 storage 依其本質就是共用的，其他節點會自己接手。

# --nodes is optional: it restricts the storage to these nodes
pvesm add synologysan mysyno \
    --syno-portal   192.0.2.10 \
    --syno-username pve-storage \
    --syno-password '<the password>' \
    --syno-location /volume1 \
    --content       images,rootdir \
    --nodes         pve1,pve2,pve3

--content takes images for VM disks and rootdir for container root filesystems and mount points, and this storage supports both. Give it both unless you have a reason not to: with images alone, a container cannot be created on it and the storage simply does not appear in the container wizard, which reads as the plugin not working. No other content type is accepted — this storage holds disks, not ISOs, templates or backups. --content 可以填 images（VM 磁碟）與 rootdir（容器的根檔案系統與掛接點），這個 storage 兩種都支援。除非有理由，兩個都給：只給 images 的話容器建不上去，而且這個 storage 根本不會出現在建立容器的畫面上，看起來就像 plugin 沒在運作。其他內容類型都不接受，這個 storage 放的是磁碟，不是 ISO、範本或備份。

pvesm add refuses immediately if the volume is not Btrfs, if the model does not support iSCSI targets or snapshots, or if the storage id would fold onto an existing storage's LUN prefix. It is meant to fail there rather than at your first snapshot. 如果那個儲存空間不是 Btrfs、機型不支援 iSCSI target 或快照、或者這個 storage id 會摺疊成與既有 storage 相同的 LUN 前置字串，pvesm add 會當場拒絕。它就是要在這裡失敗，而不是等到你第一次拍快照。

Finding the snapshots in SAN Manager在 SAN Manager 裡找到快照

SAN Manager's Snapshot page manages schedules. Expanding a LUN there shows its schedule, when the last snapshot was taken and a restore-point count —but not the snapshots themselves. To list them, go to LUN → select the LUN → Snapshot, or open Snapshot List from the Snapshot page. SAN Manager 的快照頁面管理的是「排程」。在那裡展開一個 LUN 會看到它的排程、最近一次快照的時間，以及一個快照時間點數量，但不會列出快照本身。要列出快照，請到 LUN → 選擇該 LUN → 快照，或從快照頁面開啟快照清單。

SAN Manager has no Name column. Its snapshot list shows time, consistency state, description, status and lock. The API does carry a name, and the plugin sets it to Proxmox VE's own snapshot name and matches on it —but nothing in DSM displays it. So from 0.6.5 the plugin writes the name into the description as well, as <snapshot> (Proxmox VE <storage>): SAN Manager 沒有「名稱」欄位。它的快照清單只有時間、一致性狀態、描述、狀態與鎖定。API 裡確實有 name，plugin 也會把它設成 Proxmox VE 自己的快照名稱並據以比對，但 DSM 沒有任何地方會顯示它。所以從 0.6.5 起，plugin 也會把名稱寫進描述，格式是 <快照名稱> (Proxmox VE <storage>)：

Every snapshot also carries taken_by, set to jt-pve-storage-synology. The plugin only ever lists, deletes or rolls back snapshots carrying that marker, so a schedule you set up in SAN Manager cannot be touched by a VM operation. A description is written when the snapshot is taken and never updated afterwards, so a snapshot taken by an older version keeps the text of its day. 每個快照都帶著 taken_by，值是 jt-pve-storage-synology。plugin 只會列出、刪除、倒回帶這個標記的快照，所以你自己在 SAN Manager 排的快照，不會被任何 VM 操作動到。描述是拍快照的當下寫進去的，之後不會再更新，所以舊版本拍的快照會留著當時的文字。

The same page is a quick way to confirm the storage is right: a LUN this plugin created shows Btrfs, second-level snapshot. A LUN that is not thin provisioned shows Thick Provisioning LUN does not support snapshot instead. 這一頁也可以順手確認 storage 是對的：這個 plugin 建立的 LUN 會顯示 Btrfs、秒級快照。如果那顆 LUN 不是精簡配置，DSM 會改成顯示「Thick Provisioning LUN 不支援快照功能」。

Both are 15,356,124,401,664 and 4,318,122,532,864 bytes. The ratio between the two rows is 2⁴⁰ ÷ 10¹² = 1.0995, and 15.36 ÷ 14 = 1.0971 once DSM's rounding to two significant figures is allowed for. The percentages are the quick check, and they agree: 28.12% against 28%. Proxmox VE is not consistent with itself here either —a VM's memory reads 4.00 GiB, binary and labelled as such, on the same interface. That is Proxmox VE's convention and the plugin does not second-guess it. 兩邊都是 15,356,124,401,664 與 4,318,122,532,864 位元組。兩列之間的比值是 2⁴⁰ ÷ 10¹² = 1.0995，而 15.36 ÷ 14 = 1.0971，扣掉 DSM 進位到兩位有效數字後就是同一個數。百分比是最快的檢查點，而它們一致：28.12% 對 28%。Proxmox VE 自己其實也不一致。同一個介面上，VM 的記憶體顯示為 4.00 GiB，用的是二進位單位而且標示清楚。那是 Proxmox VE 的慣例，plugin 不會去猜它想怎麼顯示。

Restricting it to certain nodes限定可用節點

nodes is Proxmox VE's own property, not a syno- one, and it works on this storage like any other. Set it at pvesm add time or later: nodes 是 Proxmox VE 自己的屬性，不是 syno- 開頭的，而它在這個 storage 上和在其他 storage 上一樣有效。可以在 pvesm add 時設定，也可以之後再設：

# restrict an existing storage
pvesm set mysyno --nodes pve1,pve2

# open it to the whole cluster again
pvesm set mysyno --delete nodes

# check
pvesm status --storage mysyno

Two reasons to use it. A node without the plugin installed will otherwise log unknown storage type on every pvestatd poll —restricting the storage is the clean way to stage a rollout. And a node with no route to the NAS's data portals has no business trying: it will fail to activate volumes rather than fail politely. 有兩個理由要用它。第一，沒有裝 plugin 的節點每一次 pvestatd 輪詢都會留下一筆「unknown storage type」的記錄，所以分批上線時先把它們排除掉最乾淨。第二，連不到 NAS 資料 portal 的節點本來就不該去嘗試：它會在啟用磁碟時失敗，而不是悄悄略過。

Every option所有選項

Removing a storage移除 storage

# 1. on EVERY node, while the storage still exists
pve-syno-reap --storage mysyno
pve-syno-reap --storage mysyno --remove

# 2. then, on one node
pvesm remove mysyno

on_delete_hook runs on one node —the one where pvesm remove was typed. The others are never told, and nothing in Proxmox VE calls deactivate_storage to clean them up. So the per-node step above is the procedure, not a formality. on_delete_hook 只在一個節點上執行，就是打 pvesm remove 的那個。其他節點不會被通知，而且 Proxmox VE 裡沒有任何東西會呼叫 deactivate_storage 去清理它們。所以上面那個逐節點步驟是真正要做的程序，不是形式。

The cleanup tool清理工具

Run it after a node crash, and on every node before removing a storage. Default is a dry run. 節點當機之後執行它，移除 storage 之前也在每個節點上執行。預設是試跑。

# show what is left behind
pve-syno-reap --storage <storage>

# then act
pve-syno-reap --storage <storage> --remove

# every synologysan storage on this node
pve-syno-reap --all --remove

pve-syno-reap clears multipath maps this node holds for LUNs the NAS no longer has, and tracking entries left by a crash. A hard-reset node never runs deactivate_volume at all, so its tracking file keeps an entry for a LUN it is not attached to —measured on a three-node cluster. pve-syno-reap 會清除本節點為 NAS 上已不存在的 LUN 所持有的 multipath map，以及當機留下的追蹤記錄。被硬重置的節點根本不會執行 deactivate_volume，所以它的追蹤檔會留著一筆對應「並未掛載」LUN 的記錄。這是在三節點叢集上量測到的。

What is a leftover, and what only looks like one哪些是殘留，哪些只是看起來像

Three cases, in the order you are likely to meet them: 三種情況，依照你可能遇到的順序：

# 1. after a node crash — the node never ran deactivate_volume, so a tracking
#    entry and a map can survive for a LUN it is no longer using
#    dry run: says what it would clear
pve-syno-reap --all
#    then act
pve-syno-reap --all --remove

# 2. before removing a storage — on EVERY node, while the storage still exists
pve-syno-reap --storage mysyno --remove
#    then once, on one node
pvesm remove mysyno

# 3. the storage is ALREADY gone from the configuration, so this is by hand.
#    CHECK BEFORE YOU FLUSH:
#    it must say SYNOLOGY, and be the wwid you expect
multipath -ll <map>
#    the Open count MUST be 0, anything else is in use
dmsetup info <map> | grep Open
#    and nothing may hold it
lsof /dev/mapper/<map>

#    only then, one map and one session at a time, and never -F:
multipathd disablequeueing map <map>
dmsetup message <map> 0 fail_if_no_path
multipath -f <map>
iscsiadm -m node -T <iqn> -p <portal> --logout
iscsiadm -m node -T <iqn> -p <portal> -o delete

pve-syno-reap clears what the NAS no longer has. A map for a LUN that still exists but has moved to another node is not in its remit, and on a healthy node it does not arise: stopping a guest or migrating it away removes the map on the source. pve-syno-reap 清的是「NAS 上已經不存在」的東西。至於「LUN 還在、只是換到別的節點」的 map，不在它的職責範圍，而在正常的節點上也不會出現：停止 guest 或把它遷移走，來源節點上的 map 就會被移除。

The discovery tool探索工具

This works today. It is read-only and safe against a production NAS. 這個現在就能用。它是唯讀的，對正式機是安全的。

It creates nothing, deletes nothing, and logs out after itself. The password is prompted for with echo off —never passed on the command line, where it would be visible in ps and in the shell history. 它不建立、不刪除任何東西，跑完會自己登出。密碼是提示輸入的，輸入時不會顯示在畫面上。絕不從命令列傳入，那會在 ps 和 shell 歷史裡看得到。

# from a .deb release, or straight out of the repository
pve-syno-api-probe --host 192.0.2.10 --user pve-storage

It reports the API set and every version range the NAS accepts, whether anti-CSRF is on, the DSM volumes and their filesystems, the LUNs and targets as they are now, the dev_attribs of an existing LUN, and a closing register of what the run settled and what still needs a write to answer. 它會回報 NAS 接受的 API 範圍與每一個版本區間、防 CSRF 是否開啟、DSM 儲存空間及其檔案系統、目前的 LUN 與 target、一顆既有 LUN 的 dev_attribs，最後給出一份驗證紀錄：這次跑完解決了什麼、還有什麼需要寫入才能回答。

About --probe-methods關於 --probe-methods

It names a LUN and snapshot uuid the NAS has never issued, so a method that exists can only refuse —and the refusal code proves it is there. DSM answers 103 for a method it does not have, and an object-not-found code for one it does. Nothing it could act on is real, but the names being sent are destructive ones, so it asks first. 它指名一個 NAS 從未發出過的 LUN 與快照 uuid，所以存在的方法只能拒絕，而拒絕的錯誤碼證明它在那裡。DSM 對沒有的方法回 103，對有的方法回「找不到物件」。它能動到的東西並不存在，但送出的畢竟是破壞性方法的名稱，所以先問過再做。

When something goes wrong故障處理

Every entry here is a failure this plugin has actually produced, with what it means and what to do. 這裡每一項都是這個 plugin 真的產生過的失敗，附上它的意思和該怎麼做。

Roadmap開發階段

Related projects相關專案

Proxmox VE storage plugins for other arrays, sharing the host-side layer and the operational rules this one inherits. 其他儲存伺服器的 Proxmox VE 儲存 plugin，與本專案共用主機端的架構與它繼承的維運規則。