okami.mom/web-extract-go
git clone https://git.nkp.pet/okami.mom/web-extract-go
5a90548
main
1[ 2 { 3"URL" :"https://sqlite.org/wal.html" , 4"FinalURL" :"https://sqlite.org/wal.html" , 5"Title" :"Write-Ahead Logging" , 6"Markdown" :"[SQLite](https://sqlite.org/index.html)\n\nSmall. Fast. Reliable. \nChoose any three.\n\n- [Home](https://sqlite.org/index.html)\n- [Menu](javascript:void%280%29)\n- [About](https://sqlite.org/about.html)\n- [Documentation](https://sqlite.org/docs.html)\n- [Download](https://sqlite.org/download.html)\n- [License](https://sqlite.org/copyright.html)\n- [Support](https://sqlite.org/support.html)\n- [Purchase](https://sqlite.org/prosupport.html)\n- [Search](javascript:void%280%29)\n\n<!--THE END-->\n\n- [About](https://sqlite.org/about.html)\n- [Documentation](https://sqlite.org/docs.html)\n- [Download](https://sqlite.org/download.html)\n- [Support](https://sqlite.org/support.html)\n- [Purchase](https://sqlite.org/prosupport.html)\n\nWrite-Ahead Logging\n\nTable Of Contents\n\n[1. Overview](https://sqlite.org/wal.html#overview)\n\n[2. How WAL Works](https://sqlite.org/wal.html#how_wal_works)\n\n[2.1. Checkpointing](https://sqlite.org/wal.html#checkpointing)\n\n[2.2. Concurrency](https://sqlite.org/wal.html#concurrency)\n\n[2.3. Performance Considerations](https://sqlite.org/wal.html#performance_considerations)\n\n[3. Activating And Configuring WAL Mode](https://sqlite.org/wal.html#activating_and_configuring_wal_mode)\n\n[3.1. Automatic Checkpoint](https://sqlite.org/wal.html#automatic_checkpoint)\n\n[3.2. Application-Initiated Checkpoints](https://sqlite.org/wal.html#application_initiated_checkpoints)\n\n[3.3. Persistence of WAL mode](https://sqlite.org/wal.html#persistence_of_wal_mode)\n\n[4. The WAL File](https://sqlite.org/wal.html#the_wal_file)\n\n[5. Read-Only Databases](https://sqlite.org/wal.html#read_only_databases)\n\n[6. Avoiding Excessively Large WAL Files](https://sqlite.org/wal.html#avoiding_excessively_large_wal_files)\n\n[7. Implementation Of Shared-Memory For The WAL-Index](https://sqlite.org/wal.html#implementation_of_shared_memory_for_the_wal_index)\n\n[8. Use of WAL Without Shared-Memory](https://sqlite.org/wal.html#use_of_wal_without_shared_memory)\n\n[9. Sometimes Queries Return SQLITE\\_BUSY In WAL Mode](https://sqlite.org/wal.html#sometimes_queries_return_sqlite_busy_in_wal_mode)\n\n[10. Backwards Compatibility](https://sqlite.org/wal.html#backwards_compatibility)\n\n[11. The WAL-Reset Bug](https://sqlite.org/wal.html#the_wal_reset_bug)\n\n[11.1. Bug Details](https://sqlite.org/wal.html#bug_details)\n\n[11.2. Low Probability Of Occurrence](https://sqlite.org/wal.html#low_probability_of_occurrence)\n\n# 1. Overview\n\nThe default method by which SQLite implements [atomic commit and rollback](https://sqlite.org/atomiccommit.html) is a [rollback journal](https://sqlite.org/lockingv3.html#rollback). Beginning with [version 3.7.0](https://sqlite.org/releaselog/3_7_0.html) (2010-07-21), a new \"Write-Ahead Log\" option (hereafter referred to as \"WAL\") is available.\n\nThere are advantages and disadvantages to using WAL instead of a rollback journal. Advantages include:\n\n[](https://sqlite.org/wal.html)\n\n1. WAL is significantly faster in most scenarios.\n2. WAL provides more concurrency as readers do not block writers and a writer does not block readers. Reading and writing can proceed concurrently.\n3. Disk I/O operations tends to be more sequential using WAL.\n4. WAL uses many fewer fsync() operations and is thus less vulnerable to problems on systems where the fsync() system call is broken.\n\nBut there are also disadvantages:\n\n1. All processes using a database must be on the same host computer; WAL does not work over a network filesystem. This is because WAL requires all processes to share a small amount of memory and processes on separate host machines obviously cannot share memory with each other.\n2. Transactions that involve changes against multiple [ATTACHed](https://sqlite.org/lang_attach.html) databases are atomic for each individual database, but are not atomic across all databases as a set.\n3. It is not possible to change the [page\\_size](https://sqlite.org/pragma.html#pragma_page_size) after entering WAL mode, either on an empty database or by using [VACUUM](https://sqlite.org/lang_vacuum.html) or by restoring from a backup using the [backup API](https://sqlite.org/backup.html). You must be in a rollback journal mode to change the page size.\n4. It is not possible to open [read-only WAL databases](https://sqlite.org/wal.html#readonly). The opening process must have write privileges for \"`-shm`\" [wal-index](https://sqlite.org/walformat.html#shm) shared memory file associated with the database, if that file exists, or else write access on the directory containing the database file if the \"`-shm`\" file does not exist. Beginning with [version 3.22.0](https://sqlite.org/releaselog/3_22_0.html) (2018-01-22), a read-only WAL-mode database file can be opened if the `-shm` and `-wal` files already exist or those files can be created or the [database is immutable](https://sqlite.org/uri.html#uriimmutable).\n5. WAL might be very slightly slower (perhaps 1% or 2% slower) than the traditional rollback-journal approach in applications that do mostly reads and seldom write.\n6. There is an additional quasi-persistent \"`-wal`\" file and \"`-shm`\" shared memory file associated with each database, which can make SQLite less appealing for use as an [application file-format](https://sqlite.org/appfileformat.html).\n7. There is the extra operation of [checkpointing](https://sqlite.org/wal.html#ckpt) which, though automatic by default, is still something that application developers need to be mindful of.\n8. WAL works best with smaller transactions. WAL does not work well for very large transactions. For transactions larger than about 100 megabytes, traditional rollback journal modes will likely be faster. For transactions in excess of a gigabyte, WAL mode may fail with an I/O or disk-full error. It is recommended that one of the rollback journal modes be used for transactions larger than a few dozen megabytes. Beginning with [version 3.11.0](https://sqlite.org/releaselog/3_11_0.html) (2016-02-15), WAL mode works as efficiently with large transactions as does rollback mode.\n\n# 2. How WAL Works\n\nThe traditional rollback journal works by writing a copy of the original unchanged database content into a separate rollback journal file and then writing changes directly into the database file. In the event of a crash or [ROLLBACK](https://sqlite.org/lang_transaction.html), the original content contained in the rollback journal is played back into the database file to revert the database file to its original state. The [COMMIT](https://sqlite.org/lang_transaction.html) occurs when the rollback journal is deleted.\n\nThe WAL approach inverts this. The original content is preserved in the database file and the changes are appended into a separate WAL file. A [COMMIT](https://sqlite.org/lang_transaction.html) occurs when a special record indicating a commit is appended to the WAL. Thus a COMMIT can happen without ever writing to the original database, which allows readers to continue operating from the original unaltered database while changes are simultaneously being committed into the WAL. Multiple transactions can be appended to the end of a single WAL file.\n\n[](https://sqlite.org/wal.html)\n\n## 2.1. Checkpointing\n\nOf course, one wants to eventually transfer all the transactions that are appended in the WAL file back into the original database. Moving the WAL file transactions back into the database is called a \"*checkpoint*\".\n\nAnother way to think about the difference between rollback and write-ahead log is that in the rollback-journal approach, there are two primitive operations, reading and writing, whereas with a write-ahead log there are now three primitive operations: reading, writing, and checkpointing.\n\nBy default, SQLite does a checkpoint automatically when the WAL file reaches a threshold size of 1000 pages. (The [SQLITE\\_DEFAULT\\_WAL\\_AUTOCHECKPOINT](https://sqlite.org/compile.html#default_wal_autocheckpoint) compile-time option can be used to specify a different default.) Applications using WAL do not have to do anything in order for these checkpoints to occur. But if they want to, applications can adjust the automatic checkpoint threshold. Or they can turn off the automatic checkpoints and run checkpoints during idle moments or in a separate thread or process.\n\n[](https://sqlite.org/wal.html)\n\n## 2.2. Concurrency\n\nWhen a read operation begins on a WAL-mode database, it first remembers the location of the last valid commit record in the WAL. Call this point the \"end mark\". Because the WAL can be growing and adding new commit records while various readers connect to the database, each reader can potentially have its own end mark. But for any particular reader, the end mark is unchanged for the duration of the transaction, thus ensuring that a single read transaction only sees the database content as it existed at a single point in time.\n\nWhen a reader needs a page of content, it first checks the WAL to see if that page appears there, and if so it pulls in the last copy of the page that occurs in the WAL prior to the reader's end mark. If no copy of the page exists in the WAL prior to the reader's end mark, then the page is read from the original database file. Readers can exist in separate processes, so to avoid forcing every reader to scan the entire WAL looking for pages (the WAL file can grow to multiple megabytes, depending on how often checkpoints are run), a data structure called the \"wal-index\" is maintained in shared memory which helps readers locate pages in the WAL quickly and with a minimum of I/O. The wal-index greatly improves the performance of readers, but the use of shared memory means that all readers must exist on the same machine. This is why the write-ahead log implementation will not work on a network filesystem.\n\nWriters merely append new content to the end of the WAL file. Because writers do nothing that would interfere with the actions of readers, writers and readers can run at the same time. However, since there is only one WAL file, there can only be one writer at a time.\n\nA checkpoint operation takes content from the WAL file and transfers it back into the original database file. A checkpoint can run concurrently with readers, however the checkpoint must stop when it reaches a page in the WAL that is past the end mark of any current reader. The checkpoint has to stop at that point because otherwise it might overwrite part of the database file that the reader is actively using. The checkpoint remembers (in the wal-index) how far it got and will resume transferring content from the WAL to the database from where it left off on the next invocation.\n\nThus a long-running read transaction can prevent a checkpointer from making progress. But presumably every read transaction will eventually end and the checkpointer will be able to continue.\n\nWhenever a write operation occurs, the writer checks how much progress the checkpointer has made, and if the entire WAL has been transferred into the database and synced and if no readers are making use of the WAL, then the writer will rewind the WAL back to the beginning and start putting new transactions at the beginning of the WAL. This mechanism prevents a WAL file from growing without bound.\n\n[](https://sqlite.org/wal.html)\n\n## 2.3. Performance Considerations\n\nWrite transactions are very fast since they only involve writing the content once (versus twice for rollback-journal transactions) and because the writes are all sequential. Further, syncing the content to the disk is not required, as long as the application is willing to sacrifice durability following a power loss or hard reboot. (Writers sync the WAL on every transaction commit if [PRAGMA synchronous](https://sqlite.org/pragma.html#pragma_synchronous) is set to FULL but omit this sync if [PRAGMA synchronous](https://sqlite.org/pragma.html#pragma_synchronous) is set to NORMAL.)\n\nOn the other hand, read performance deteriorates as the WAL file grows in size since each reader must check the WAL file for the content and the time needed to check the WAL file is proportional to the size of the WAL file. The wal-index helps find content in the WAL file much faster, but performance still falls off with increasing WAL file size. Hence, to maintain good read performance it is important to keep the WAL file size down by running checkpoints at regular intervals.\n\nCheckpointing does require sync operations in order to avoid the possibility of database corruption following a power loss or hard reboot. The WAL must be synced to persistent storage prior to moving content from the WAL into the database and the database file must be synced prior to resetting the WAL. Checkpoint also requires more seeking. The checkpointer makes an effort to do as many sequential page writes to the database as it can (the pages are transferred from WAL to database in ascending order) but even then there will typically be many seek operations interspersed among the page writes. These factors combine to make checkpoints slower than write transactions.\n\nThe default strategy is to allow successive write transactions to grow the WAL until the WAL becomes about 1000 pages in size, then to run a checkpoint operation for each subsequent COMMIT until the WAL is reset to be smaller than 1000 pages. By default, the checkpoint will be run automatically by the same thread that does the COMMIT that pushes the WAL over its size limit. This has the effect of causing most COMMIT operations to be very fast but an occasional COMMIT (those that trigger a checkpoint) to be much slower. If that effect is undesirable, then the application can disable automatic checkpointing and run the periodic checkpoints in a separate thread, or separate process. (Links to commands and interfaces to accomplish this are [shown below](https://sqlite.org/wal.html#how_to_checkpoint).)\n\nNote that with [PRAGMA synchronous](https://sqlite.org/pragma.html#pragma_synchronous) set to NORMAL, the checkpoint is the only operation to issue an I/O barrier or sync operation (fsync() on unix or FlushFileBuffers() on windows). If an application therefore runs checkpoint in a separate thread or process, the main thread or process that is doing database queries and updates will never block on a sync operation. This helps to prevent \"latch-up\" in applications running on a busy disk drive. The downside to this configuration is that transactions are no longer durable and might rollback following a power failure or hard reset.\n\nNotice too that there is a tradeoff between average read performance and average write performance. To maximize the read performance, one wants to keep the WAL as small as possible and hence run checkpoints frequently, perhaps as often as every COMMIT. To maximize write performance, one wants to amortize the cost of each checkpoint over as many writes as possible, meaning that one wants to run checkpoints infrequently and let the WAL grow as large as possible before each checkpoint. The decision of how often to run checkpoints may therefore vary from one application to another depending on the relative read and write performance requirements of the application. The default strategy is to run a checkpoint once the WAL reaches 1000 pages and this strategy seems to work well in test applications on workstations, but other strategies might work better on different platforms or for different workloads.\n\n# 3. Activating And Configuring WAL Mode\n\nAn SQLite database connection defaults to [journal\\_mode=DELETE](https://sqlite.org/pragma.html#pragma_journal_mode). To convert to WAL mode, use the following pragma:\n\n> ```\n> PRAGMA journal_mode=WAL;\n> ```\n\nThe journal\\_mode pragma returns a string which is the new journal mode. On success, the pragma will return the string \"`wal`\". If the conversion to WAL could not be completed (for example, if the [VFS](https://sqlite.org/vfs.html) does not support the necessary shared-memory primitives) then the journaling mode will be unchanged and the string returned from the primitive will be the prior journaling mode (for example \"`delete`\").[](https://sqlite.org/wal.html)\n\n## 3.1. Automatic Checkpoint\n\nBy default, SQLite will automatically checkpoint whenever a [COMMIT](https://sqlite.org/lang_transaction.html) occurs that causes the WAL file to be 1000 pages or more in size, or when the last database connection on a database file closes. The default configuration is intended to work well for most applications. But programs that want more control can force a checkpoint using the [wal\\_checkpoint pragma](https://sqlite.org/pragma.html#pragma_wal_checkpoint) or by calling the [sqlite3\\_wal\\_checkpoint()](https://sqlite.org/c3ref/wal_checkpoint.html) C interface. The automatic checkpoint threshold can be changed or automatic checkpointing can be completely disabled using the [wal\\_autocheckpoint pragma](https://sqlite.org/pragma.html#pragma_wal_autocheckpoint) or by calling the [sqlite3\\_wal\\_autocheckpoint()](https://sqlite.org/c3ref/wal_autocheckpoint.html) C interface. A program can also use [sqlite3\\_wal\\_hook()](https://sqlite.org/c3ref/wal_hook.html) to register a callback to be invoked whenever any transaction commits to the WAL. This callback can then invoke [sqlite3\\_wal\\_checkpoint()](https://sqlite.org/c3ref/wal_checkpoint.html) or [sqlite3\\_wal\\_checkpoint\\_v2()](https://sqlite.org/c3ref/wal_checkpoint_v2.html) based on whatever criteria it thinks is appropriate. (The automatic checkpoint mechanism is implemented as a simple wrapper around [sqlite3\\_wal\\_hook()](https://sqlite.org/c3ref/wal_hook.html).)\n\n## 3.2. Application-Initiated Checkpoints\n\nAn application can initiate a checkpoint using any writable database connection on the database simply by invoking [sqlite3\\_wal\\_checkpoint()](https://sqlite.org/c3ref/wal_checkpoint.html) or [sqlite3\\_wal\\_checkpoint\\_v2()](https://sqlite.org/c3ref/wal_checkpoint_v2.html). There are three subtypes of checkpoints that vary in their aggressiveness: PASSIVE, FULL, and RESTART. The default checkpoint style is PASSIVE, which does as much work as it can without interfering with other database connections, and which might not run to completion if there are concurrent readers or writers. All checkpoints initiated by [sqlite3\\_wal\\_checkpoint()](https://sqlite.org/c3ref/wal_checkpoint.html) and by the automatic checkpoint mechanism are PASSIVE. FULL and RESTART checkpoints try harder to run the checkpoint to completion and can only be initiated by a call to [sqlite3\\_wal\\_checkpoint\\_v2()](https://sqlite.org/c3ref/wal_checkpoint_v2.html). See the [sqlite3\\_wal\\_checkpoint\\_v2()](https://sqlite.org/c3ref/wal_checkpoint_v2.html) documentation for additional information on FULL and RESET checkpoints.\n\n## 3.3. Persistence of WAL mode\n\nUnlike the other journaling modes, [PRAGMA journal\\_mode=WAL](https://sqlite.org/pragma.html#pragma_journal_mode) is persistent. If a process sets WAL mode, then closes and reopens the database, the database will come back in WAL mode. In contrast, if a process sets (for example) PRAGMA journal\\_mode=TRUNCATE and then closes and reopens the database will come back up in the default rollback mode of DELETE rather than the previous TRUNCATE setting.\n\nThe persistence of WAL mode means that applications can be converted to using SQLite in WAL mode without making any changes to the application itself. One has merely to run \"`PRAGMA journal_mode=WAL;`\" on the database file(s) using the [command-line shell](https://sqlite.org/cli.html) or other utility, then restart the application.\n\nThe WAL journal mode will be set on all connections to the same database file if it is set on any one connection.\n\n[](https://sqlite.org/wal.html)\n\n# 4. The WAL File\n\nWhile a [database connection](https://sqlite.org/c3ref/sqlite3.html) is open on a WAL-mode database, SQLite maintains an extra journal file called a \"Write Ahead Log\" or \"WAL File\". The name of this file on disk is usually the name of the database file with an extra \"`-wal`\" suffix, though different naming rules may apply if SQLite is compiled with [SQLITE\\_ENABLE\\_8\\_3\\_NAMES](https://sqlite.org/compile.html#enable_8_3_names).\n\nThe WAL file exists for as long as any [database connection](https://sqlite.org/c3ref/sqlite3.html) has the database open. Usually, the WAL file is deleted automatically when the last connection to the database closes. However, if the last process to have the database open exits without cleanly shutting down the database connection, or if the [SQLITE\\_FCNTL\\_PERSIST\\_WAL](https://sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal) [file control](https://sqlite.org/c3ref/file_control.html) is used, then the WAL file might be retained on disk after all connections to the database have been closed. The WAL file is part of the persistent state of the database and should be kept with the database if the database is copied or moved. If a database file is separated from its WAL file, then transactions that were previously committed to the database might be lost, or the database file might become corrupted. The only safe way to remove a WAL file is to open the database file using one of the [sqlite3\\_open()](https://sqlite.org/c3ref/open.html) interfaces then immediately close the database using [sqlite3\\_close()](https://sqlite.org/c3ref/close.html).\n\nThe [WAL file format](https://sqlite.org/fileformat2.html#walformat) is precisely defined and is cross-platform.[](https://sqlite.org/wal.html)\n\n# 5. Read-Only Databases\n\nOlder versions of SQLite could not read a WAL-mode database that was read-only. In other words, write access was required in order to read a WAL-mode database. This constraint was relaxed beginning with SQLite [version 3.22.0](https://sqlite.org/releaselog/3_22_0.html) (2018-01-22).\n\nOn newer versions of SQLite, a WAL-mode database on read-only media, or a WAL-mode database that lacks write permission, can still be read as long as one or more of the following conditions are met:\n\n1. The `-shm` and `-wal` files already exist and are readable.\n2. There is write permission on the directory containing the database so that the `-shm` and `-wal` files can be created.\n3. The database connection is opened using the [immutable query parameter](https://sqlite.org/uri.html#uriimmutable).\n\nEven though it is possible to open a read-only WAL-mode database, it is good practice to convert the database to [PRAGMA journal\\_mode=DELETE](https://sqlite.org/pragma.html#pragma_journal_mode) prior to burning an SQLite database image onto read-only media.\n\n[](https://sqlite.org/wal.html)\n\n# 6. Avoiding Excessively Large WAL Files\n\nIn normal cases, new content is appended to the WAL file until the WAL file accumulates about 1000 pages (and is thus about 4MB in size) at which point a checkpoint is automatically run and the WAL file is recycled. The checkpoint does not normally truncate the WAL file (unless the [journal\\_size\\_limit pragma](https://sqlite.org/pragma.html#pragma_journal_size_limit) is set). Instead, it merely causes SQLite to start overwriting the WAL file from the beginning. This is done because it is normally faster to overwrite an existing file than to append. When the last connection to a database closes, that connection does one last checkpoint and then deletes the WAL and its associated shared-memory file, to clean up the disk.\n\nSo in the vast majority of cases, applications need not worry about the WAL file at all. SQLite will automatically take care of it. But it is possible to get SQLite into a state where the WAL file will grow without bound, causing excess disk space usage and slow query speeds. The following bullets enumerate some of the ways that this can happen and how to avoid them.\n\n- **Disabling the automatic checkpoint mechanism.** In its default configuration, SQLite will checkpoint the WAL file at the conclusion of any transaction when the WAL file is more than 1000 pages long. However, compile-time and run-time options exist that can disable or defer this automatic checkpoint. If an application disables the automatic checkpoint, then there is nothing to prevent the WAL file from growing excessively.\n- **Checkpoint starvation.** A checkpoint is only able to run to completion, and reset the WAL file, if there are no other database connections using the WAL file. If another connection has a read transaction open, then the checkpoint cannot reset the WAL file because doing so might delete content out from under the reader. The checkpoint will do as much work as it can without upsetting the reader, but it cannot run to completion. The checkpoint will start up again where it left off after the next write transaction. This repeats until some checkpoint is able to complete.\n \n However, if a database has many concurrent overlapping readers and there is always at least one active reader, then no checkpoints will be able to complete and hence the WAL file will grow without bound.\n \n This scenario can be avoided by ensuring that there are \"reader gaps\": times when no processes are reading from the database and that checkpoints are attempted during those times. In applications with many concurrent readers, one might also consider running manual checkpoints with the [SQLITE\\_CHECKPOINT\\_RESTART](https://sqlite.org/c3ref/c_checkpoint_full.html) or [SQLITE\\_CHECKPOINT\\_TRUNCATE](https://sqlite.org/c3ref/c_checkpoint_full.html) option which will ensure that the checkpoint runs to completion before returning. The disadvantage of using [SQLITE\\_CHECKPOINT\\_RESTART](https://sqlite.org/c3ref/c_checkpoint_full.html) and [SQLITE\\_CHECKPOINT\\_TRUNCATE](https://sqlite.org/c3ref/c_checkpoint_full.html) is that readers might block while the checkpoint is running.\n- **Very large write transactions.** A checkpoint can only complete when no other transactions are running, which means the WAL file cannot be reset in the middle of a write transaction. So a large change to a large database might result in a large WAL file. The WAL file will be checkpointed once the write transaction completes (assuming there are no other readers blocking it) but in the meantime, the file can grow very big.\n \n As of SQLite [version 3.11.0](https://sqlite.org/releaselog/3_11_0.html) (2016-02-15), the WAL file for a single transaction should be proportional in size to the transaction itself. Pages that are changed by the transaction should only be written into the WAL file once. However, with older versions of SQLite, the same page might be written into the WAL file multiple times if the transaction grows larger than the page cache.\n\n# 7. Implementation Of Shared-Memory For The WAL-Index\n\nThe [wal-index](https://sqlite.org/walformat.html#shm) is implemented using an ordinary file that is mmapped for robustness. Early (pre-release) implementations of WAL mode stored the wal-index in volatile shared-memory, such as files created in /dev/shm on Linux or /tmp on other unix systems. The problem with that approach is that processes with a different root directory (changed via [chroot](http://en.wikipedia.org/wiki/Chroot)) will see different files and hence use different shared memory areas, leading to database corruption. Other methods for creating nameless shared memory blocks are not portable across the various flavors of unix. And we could not find any method to create nameless shared memory blocks on windows. The only way we have found to guarantee that all processes accessing the same database file use the same shared memory is to create the shared memory by mmapping a file in the same directory as the database itself.\n\nUsing an ordinary disk file to provide shared memory has the disadvantage that it might actually do unnecessary disk I/O by writing the shared memory to disk. However, the developers do not think this is a major concern since the wal-index rarely exceeds 32 KiB in size and is never synced. Furthermore, the wal-index backing file is deleted when the last database connection disconnects, which often prevents any real disk I/O from ever happening.\n\nSpecialized applications for which the default implementation of shared memory is unacceptable can devise alternative methods via a custom [VFS](https://sqlite.org/vfs.html). For example, if it is known that a particular database will only be accessed by threads within a single process, the wal-index can be implemented using heap memory instead of true shared memory.\n\n[](https://sqlite.org/wal.html)\n\n# 8. Use of WAL Without Shared-Memory\n\nBeginning in SQLite [version 3.7.4](https://sqlite.org/releaselog/3_7_4.html) (2010-12-07), WAL databases can be created, read, and written even if shared memory is unavailable as long as the [locking\\_mode](https://sqlite.org/pragma.html#pragma_locking_mode) is set to EXCLUSIVE before the first attempted access. In other words, a process can interact with a WAL database without using shared memory if that process is guaranteed to be the only process accessing the database. This feature allows WAL databases to be created, read, and written by legacy [VFSes](https://sqlite.org/vfs.html) that lack the \"version 2\" shared-memory methods xShmMap, xShmLock, xShmBarrier, and xShmUnmap on the [sqlite3\\_io\\_methods](https://sqlite.org/c3ref/io_methods.html) object.\n\nIf [EXCLUSIVE locking mode](https://sqlite.org/pragma.html#pragma_locking_mode) is set prior to the first WAL-mode database access, then SQLite never attempts to call any of the shared-memory methods and hence no shared-memory wal-index is ever created. In that case, the database connection remains in EXCLUSIVE mode as long as the journal mode is WAL; attempts to change the locking mode using \"`PRAGMA locking_mode=NORMAL;`\" are no-ops. The only way to change out of EXCLUSIVE locking mode is to first change out of WAL journal mode.\n\nIf NORMAL locking mode is in effect for the first WAL-mode database access, then the shared-memory wal-index is created. This means that the underlying VFS must support the \"version 2\" shared-memory. If the VFS does not support shared-memory methods, then the attempt to open a database that is already in WAL mode, or the attempt to convert a database into WAL mode, will fail. As long as exactly one connection is using a shared-memory wal-index, the locking mode can be changed freely between NORMAL and EXCLUSIVE. It is only when the shared-memory wal-index is omitted, when the locking mode is EXCLUSIVE prior to the first WAL-mode database access, that the locking mode is stuck in EXCLUSIVE.\n\n[](https://sqlite.org/wal.html)\n\n# 9. Sometimes Queries Return SQLITE\\_BUSY In WAL Mode\n\nThe [second advantage of WAL-mode](https://sqlite.org/wal.html#advantages) is that writers do not block readers and readers do not block writers. This is mostly true. But there are some obscure cases where a query against a WAL-mode database can return [SQLITE\\_BUSY](https://sqlite.org/rescode.html#busy), so applications should be prepared for that happenstance.\n\nCases where a query against a WAL-mode database can return [SQLITE\\_BUSY](https://sqlite.org/rescode.html#busy) include the following:\n\n- If another database connection has the database mode open in [exclusive locking mode](https://sqlite.org/pragma.html#pragma_locking_mode) then all queries against the database will return [SQLITE\\_BUSY](https://sqlite.org/rescode.html#busy). Both Chrome and Firefox open their database files in exclusive locking mode, so attempts to read Chrome or Firefox databases while the applications are running will run into this problem, for example.\n- When the last connection to a particular database is closing, that connection will acquire an exclusive lock for a short time while it cleans up the WAL and shared-memory files. If a separate attempt is made to open and query the database while the first connection is still in the middle of its cleanup process, the second connection might get an [SQLITE\\_BUSY](https://sqlite.org/rescode.html#busy) error.\n- If the last connection to a database crashed, then the first new connection to open the database will start a recovery process. An exclusive lock is held during recovery. So if a third database connection tries to jump in and query while the second connection is running recovery, the third connection will get an [SQLITE\\_BUSY](https://sqlite.org/rescode.html#busy) error.\n\n[](https://sqlite.org/wal.html)\n\n# 10. Backwards Compatibility\n\nThe database file format is unchanged for WAL mode. However, the WAL file and the [wal-index](https://sqlite.org/walformat.html#shm) are new concepts and so older versions of SQLite will not know how to recover a crashed SQLite database that was operating in WAL mode when the crash occurred. To prevent older versions of SQLite (prior to version 3.7.0, 2010-07-22) from trying to recover a WAL-mode database (and making matters worse) the database file format version numbers (bytes 18 and 19 in the [database header](https://sqlite.org/fileformat2.html#database_header)) are increased from 1 to 2 in WAL mode. Thus, if an older version of SQLite attempts to connect to an SQLite database that is operating in WAL mode, it will report an error along the lines of \"file is encrypted or is not a database\".\n\nOne can explicitly change out of WAL mode using a pragma such as this:\n\n> ```\n> PRAGMA journal_mode=DELETE;\n> ```\n\nDeliberately changing out of WAL mode changes the database file format version numbers back to 1 so that older versions of SQLite can once again access the database file.\n\n[](https://sqlite.org/wal.html)\n\n# 11. The WAL-Reset Bug\n\nOn 2026-03-03, one of the SQLite developers (Dan) found and fixed a bug that could, in rare cases, lead to database corruption. We call this the \"WAL-reset bug\". Key points:\n\n- The bug is likely present in all version of SQLite from 3.7.0 (2010-07-21) through 3.51.2 (2026-01-09). It is fixed in version 3.51.3 (2026-03-13) and later. Backports of the fix are available for some earlier releases: [3.44.6](https://sqlite.org/src/info/863c171b76cd36e0) and [3.50.7](https://sqlite.org/src/info/c7facf7ac58d2cda).\n- The bug only affects databases in WAL mode when there are two or more database connections open on the same file, in separate threads or processes, and when those two connections attempt to write or checkpoint at the same instant.\n- The bug is a data race with tight timing constraints and that requires an unusual usage pattern. It is unlikely to occur in common use. The developers were unable to reproduce the bug organically and had to add special testing logic to SQLite that deliberately triggers the circumstances of the the bug in order to verify that the issue has been fixed.\n\n## 11.1. Bug Details\n\nThis is what happens:\n\n1. One connection does a [checkpoint](https://sqlite.org/wal.html#ckpt). This first checkpoint must be complete. In other words, the checkpoint must successfully copy all content from the WAL file back into the database and leave the WAL file in a state where it can potentially be reset.\n2. Shortly after the first checkpoint completes, a second checkpoint is started.\n3. While the second checkpoint from step 2 is starting up, another database connection commits a transaction that resets the WAL file and writes new content into the beginning of the WAL file.\n4. Due to a data race, the second checkpoint from step 2 does not realize that the WAL file has been reset by the transaction commit in step 3. The second checkpoint leaves a field in the header of the WAL-Index set incorrectly. That field indicates that part of the WAL file has already been checkpointed, when in fact it has not been.\n5. Additional transactions are committed to increase the number of pages in the WAL file to be more than were present for the first checkpoint from step 1.\n6. Later when a third checkpoint occurs, the third checkpoint skips all or part of the transaction that was written in step 3. Thus parts of the transaction from step 3 never reach the database file, and the database file goes corrupt.\n\n## 11.2. Low Probability Of Occurrence\n\nIn order for this bug to happen, many details must align at just the right moment. So much so that the SQLite developers were unable to reproduce the bug organically in the lab. The only way the developers have been able to cause the malfunction was to modify the SQLite sources to invoke a callback controlled by [sqlite3\\_test\\_control()](https://sqlite.org/c3ref/test_control.html) that enables a test program or script to trigger the write transaction of step 3 at just the right moment during the second checkpoint. Without that code hack, the problem has never been observed during development and testing.\n\n**UPDATE: 2026-08-24 \u2192** Phil Eaton has devised a reproducer that does not involve using the special testing code hack. See [Another look at SQLite's WAL-Reset bug](https://theconsensus.dev/p/2026/08/23/another-look-at-sqlite-wal-reset.html) for details.\n\nThis bug, though rare, does have serious consequences, and so application developers should upgrade to a version of SQLite that fixes the problem. However, this is not an emergency. Based on available telemetry, the occurrence rate of this problem in the wild appears to be less than or equal to the expected occurrence rate of SSD malfunctions and/or cosmic-ray hits. So even if you are running an unpatched version of SQLite, and unless you are doing something really unusual, you are unlikely to ever encounter this problem.\n\n*This page was last updated on 2026-08-25 19:42:39Z*" , 7"Hash" :"b034bcd3d176503f141d30eb7cad03d326b62b1743080ab0be8bf4b016470823" 8 }, 9 { 10"URL" :"https://pkg.go.dev/errors" , 11"FinalURL" :"https://pkg.go.dev/errors" , 12"Title" :"errors package - errors - Go Packages" , 13"Markdown" :"# errors\n\npackage standard library\n\n[go1.27.1](https://pkg.go.dev/errors?tab=versions)\n\nLatest\n\nLatest\n\nWarning\n\nThis package is not in the latest version of its module.\n\n[Go to latest](https://pkg.go.dev/errors)\n\nPublished: Sep 1, 2026 License: [BSD-3-Clause](https://pkg.go.dev/errors?tab=licenses) [Imports: 2](https://pkg.go.dev/errors?tab=imports) [Imported by: 1,651,855](https://pkg.go.dev/errors?tab=importedby)\n\nMain Versions Licenses Imports Imported By\n\n## Details\n\n- checked Valid [go.mod](https://cs.opensource.google/go/go/+/go1.27.1:src/go.mod) file\n \n The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.\n- checked Redistributable license\n \n Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.\n- checked Tagged version\n \n Modules with tagged versions give importers more predictable builds.\n- checked Stable version\n \n When a project reaches major version v1 it is considered stable.\n- [Learn more about best practices](https://pkg.go.dev/about#best-practices)\n\n## Repository\n\n[cs.opensource.google/go/go](https://cs.opensource.google/go/go \"https://cs.opensource.google/go/go\")\n\n## Links\n\n- [Report a Vulnerability](https://go.dev/security/policy \"Report security issues in the Go standard library and sub-repositories\")\n\n## Documentation [\u00b6](https://pkg.go.dev/errors#section-documentation \"Go to Documentation\")\n\n### Overview [\u00b6](https://pkg.go.dev/errors#pkg-overview \"Go to Overview\")\n\nPackage errors implements functions to manipulate errors.\n\nThe [New](https://pkg.go.dev/errors#New) function creates errors whose only content is a text message.\n\nAn error e wraps another error if e's type has one of the methods\n\n```\nUnwrap() error\nUnwrap() []error\n```\n\nIf e.Unwrap() returns a non-nil error w or a slice containing w, then we say that e wraps w. A nil error returned from e.Unwrap() indicates that e does not wrap any error. It is invalid for an Unwrap method to return an \\[]error containing a nil error value.\n\nAn easy way to create wrapped errors is to call [fmt.Errorf](https://pkg.go.dev/fmt#Errorf) and apply the %w verb to the error argument:\n\n```\nwrapsErr := fmt.Errorf(\"... %w ...\", ..., err, ...)\n```\n\nSuccessive unwrapping of an error creates a tree. The [Is](https://pkg.go.dev/errors#Is) and [As](https://pkg.go.dev/errors#As) functions inspect an error's tree by examining first the error itself followed by the tree of each of its children in turn (pre-order, depth-first traversal).\n\nSee [https://go.dev/blog/go1.13-errors](https://go.dev/blog/go1.13-errors) for a deeper discussion of the philosophy of wrapping and when to wrap.\n\n[Is](https://pkg.go.dev/errors#Is) examines the tree of its first argument looking for an error that matches the second. It reports whether it finds a match. It should be used in preference to simple equality checks:\n\n```\nif errors.Is(err, fs.ErrExist)\n```\n\nis preferable to\n\n```\nif err == fs.ErrExist\n```\n\nbecause the former will succeed if err wraps [io/fs.ErrExist](https://pkg.go.dev/io/fs#ErrExist).\n\n[AsType](https://pkg.go.dev/errors#AsType) examines the tree of its argument looking for an error whose type matches its type argument. If it succeeds, it returns the corresponding value of that type and true. Otherwise, it returns the zero value of that type and false. The form\n\n```\nif perr, ok := errors.AsType[*fs.PathError](err); ok {\n\tfmt.Println(perr.Path)\n}\n```\n\nis preferable to\n\n```\nif perr, ok := err.(*fs.PathError); ok {\n\tfmt.Println(perr.Path)\n}\n```\n\nbecause the former will succeed if err wraps an [\\*io/fs.PathError](https://pkg.go.dev/io/fs#PathError).\n\nExample [\u00b6](https://pkg.go.dev/errors#example-package \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"fmt\"\n\t\"time\"\n)\n\n// MyError is an error implementation that includes a time and message.\ntype MyError struct {\n\tWhen time.Time\n\tWhat string\n}\n\nfunc (e MyError) Error() string {\n\treturn fmt.Sprintf(\"%v: %v\", e.When, e.What)\n}\n\nfunc oops() error {\n\treturn MyError{\n\t\ttime.Date(1989, 3, 15, 22, 30, 0, 0, time.UTC),\n\t\t\"the file system has gone away\",\n\t}\n}\n\nfunc main() {\n\tif err := oops(); err != nil {\n\t\tfmt.Println(err)\n\t}\n}\n```\n\n```\nOutput:\n1989-03-15 22:30:00 +0000 UTC: the file system has gone away\n```\n\nShare Format Run\n\n### Index [\u00b6](https://pkg.go.dev/errors#pkg-index \"Go to Index\")\n\n- [Variables](https://pkg.go.dev/errors#pkg-variables)\n- [func As(err error, target any) bool](https://pkg.go.dev/errors#As)\n- [func AsType\\[E error\\](err error) (E, bool)](https://pkg.go.dev/errors#AsType)\n- [func Is(err, target error) bool](https://pkg.go.dev/errors#Is)\n- [func Join(errs ...error) error](https://pkg.go.dev/errors#Join)\n- [func New(text string) error](https://pkg.go.dev/errors#New)\n- [func Unwrap(err error) error](https://pkg.go.dev/errors#Unwrap)\n\n### Examples [\u00b6](https://pkg.go.dev/errors#pkg-examples \"Go to Examples\")\n\n- [Package](https://pkg.go.dev/errors#example-package)\n- [As](https://pkg.go.dev/errors#example-As)\n- [As (Custom\\_match)](https://pkg.go.dev/errors#example-As-Custom_match)\n- [AsType](https://pkg.go.dev/errors#example-AsType)\n- [AsType (Custom\\_match)](https://pkg.go.dev/errors#example-AsType-Custom_match)\n- [Is](https://pkg.go.dev/errors#example-Is)\n- [Is (Custom\\_match)](https://pkg.go.dev/errors#example-Is-Custom_match)\n- [Join](https://pkg.go.dev/errors#example-Join)\n- [New](https://pkg.go.dev/errors#example-New)\n- [New (Errorf)](https://pkg.go.dev/errors#example-New-Errorf)\n- [New (Unique)](https://pkg.go.dev/errors#example-New-Unique)\n- [Unwrap](https://pkg.go.dev/errors#example-Unwrap)\n\n### Constants [\u00b6](https://pkg.go.dev/errors#pkg-constants \"Go to Constants\")\n\nThis section is empty.\n\n### Variables [\u00b6](https://pkg.go.dev/errors#pkg-variables \"Go to Variables\")\n\n[View Source](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/errors.go;l=90)\n\n```\nvar ErrUnsupported = New(\"unsupported operation\")\n```\n\nErrUnsupported indicates that a requested operation cannot be performed, because it is unsupported. For example, a call to [os.Link](https://pkg.go.dev/os#Link) when using a file system that does not support hard links.\n\nFunctions and methods should not return this error but should instead return an error including appropriate context that satisfies\n\n```\nerrors.Is(err, errors.ErrUnsupported)\n```\n\neither by directly wrapping ErrUnsupported or by implementing an [Is](https://pkg.go.dev/errors#Is) method.\n\nFunctions and methods should document the cases in which an error wrapping this will be returned.\n\n### Functions [\u00b6](https://pkg.go.dev/errors#pkg-functions \"Go to Functions\")\n\n#### func [As](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/wrap.go;l=102) [\u00b6](https://pkg.go.dev/errors#As \"Go to As\") added in go1.13\n\n```\nfunc As(err error, target any) bool\n```\n\nAs finds the first error in err's tree that matches target, and if one is found, sets target to that error value and returns true. Otherwise, it returns false.\n\nFor most uses, prefer [AsType](https://pkg.go.dev/errors#AsType). As is equivalent to [AsType](https://pkg.go.dev/errors#AsType) but sets its target argument rather than returning the matching error and doesn't require its target argument to implement error.\n\nThe tree consists of err itself, followed by the errors obtained by repeatedly calling its Unwrap() error or Unwrap() \\[]error method. When err wraps multiple errors, As examines err followed by a depth-first traversal of its children.\n\nAn error matches target if the error's concrete value is assignable to the value pointed to by target, or if the error has a method As(any) bool such that As(target) returns true. In the latter case, the As method is responsible for setting target.\n\nAn error type might provide an As method so it can be treated as if it were a different error type.\n\nAs panics if target is not a non-nil pointer to either a type that implements error, or to any interface type.\n\nExample [\u00b6](https://pkg.go.dev/errors#example-As \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n\t\"os\"\n)\n\nfunc main() {\n\tif _, err := os.Open(\"non-existing\"); err != nil {\n\t\tvar pathError *fs.PathError\n\t\tif errors.As(err, &pathError) {\n\t\t\tfmt.Println(\"Failed at path:\", pathError.Path)\n\t\t} else {\n\t\t\tfmt.Println(err)\n\t\t}\n\t}\n\n}\n```\n\n```\nOutput:\nFailed at path: non-existing\n```\n\nShare Format Run\n\nExample (Custom\\_match) [\u00b6](https://pkg.go.dev/errors#example-As-Custom_match \"Go to Example (Custom_match)\")\n\nCustom errors can implement a method \"As(any) bool\" to match against other error types, overriding the default matching of [errors.As](https://pkg.go.dev/errors#As).\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n)\n\ntype MyAsError struct {\n\terr string\n}\n\nfunc (e MyAsError) Error() string {\n\treturn e.err\n}\nfunc (e MyAsError) As(target any) bool {\n\tpe, ok := target.(**fs.PathError)\n\tif !ok {\n\t\treturn false\n\t}\n\t*pe = &fs.PathError{\n\t\tOp: \"custom\",\n\t\tPath: \"/\",\n\t\tErr: errors.New(e.err),\n\t}\n\treturn true\n}\n\nfunc main() {\n\tvar err error = MyAsError{\"an error\"}\n\tfmt.Println(\"Error:\", err)\n\tfmt.Printf(\"TypeOf err: %T\\n\", err)\n\n\tvar pathError *fs.PathError\n\tok := errors.As(err, &pathError)\n\tfmt.Println(\"Error as fs.PathError:\", ok)\n\tfmt.Println(\"fs.PathError:\", pathError)\n\n}\n```\n\n```\nOutput:\nError: an error\nTypeOf err: errors_test.MyAsError\nError as fs.PathError: true\nfs.PathError: custom /: an error\n```\n\nShare Format Run\n\n#### func [AsType](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/wrap.go;l=167) [\u00b6](https://pkg.go.dev/errors#AsType \"Go to AsType\") added in go1.26.0\n\n```\nfunc AsType[E error](err error) (E, bool)\n```\n\nAsType finds the first error in err's tree that matches the type E, and if one is found, returns that error value and true. Otherwise, it returns the zero value of E and false.\n\nThe tree consists of err itself, followed by the errors obtained by repeatedly calling its Unwrap() error or Unwrap() \\[]error method. When err wraps multiple errors, AsType examines err followed by a depth-first traversal of its children.\n\nAn error err matches the type E if the type assertion err.(E) holds, or if the error has a method As(any) bool such that err.As(target) returns true when target is a non-nil \\*E. In the latter case, the As method is responsible for setting target.\n\nExample [\u00b6](https://pkg.go.dev/errors#example-AsType \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n\t\"os\"\n)\n\nfunc main() {\n\tif _, err := os.Open(\"non-existing\"); err != nil {\n\t\tif pathError, ok := errors.AsType[*fs.PathError](err); ok {\n\t\t\tfmt.Println(\"Failed at path:\", pathError.Path)\n\t\t} else {\n\t\t\tfmt.Println(err)\n\t\t}\n\t}\n}\n```\n\n```\nOutput:\nFailed at path: non-existing\n```\n\nShare Format Run\n\nExample (Custom\\_match) [\u00b6](https://pkg.go.dev/errors#example-AsType-Custom_match \"Go to Example (Custom_match)\")\n\nCustom errors can implement a method \"As(any) bool\" to match against other error types, overriding the default matching of [errors.AsType](https://pkg.go.dev/errors#AsType).\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n)\n\ntype MyAsError struct {\n\terr string\n}\n\nfunc (e MyAsError) Error() string {\n\treturn e.err\n}\nfunc (e MyAsError) As(target any) bool {\n\tpe, ok := target.(**fs.PathError)\n\tif !ok {\n\t\treturn false\n\t}\n\t*pe = &fs.PathError{\n\t\tOp: \"custom\",\n\t\tPath: \"/\",\n\t\tErr: errors.New(e.err),\n\t}\n\treturn true\n}\n\nfunc main() {\n\tvar err error = MyAsError{\"an error\"}\n\tfmt.Println(\"Error:\", err)\n\tfmt.Printf(\"TypeOf err: %T\\n\", err)\n\n\tpathError, ok := errors.AsType[*fs.PathError](err)\n\tfmt.Println(\"Error as fs.PathError:\", ok)\n\tfmt.Println(\"fs.PathError:\", pathError)\n\n}\n```\n\n```\nOutput:\nError: an error\nTypeOf err: errors_test.MyAsError\nError as fs.PathError: true\nfs.PathError: custom /: an error\n```\n\nShare Format Run\n\n#### func [Is](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/wrap.go;l=45) [\u00b6](https://pkg.go.dev/errors#Is \"Go to Is\") added in go1.13\n\n```\nfunc Is(err, target error) bool\n```\n\nIs reports whether any error in err's tree matches target. The target must be comparable.\n\nThe tree consists of err itself, followed by the errors obtained by repeatedly calling its Unwrap() error or Unwrap() \\[]error method. When err wraps multiple errors, Is examines err followed by a depth-first traversal of its children.\n\nAn error is considered to match a target if it is equal to that target or if it implements a method Is(error) bool such that Is(target) returns true.\n\nAn error type might provide an Is method so it can be treated as equivalent to an existing error. For example, if MyError defines\n\n```\nfunc (m MyError) Is(target error) bool { return target == fs.ErrExist }\n```\n\nthen Is(MyError{}, fs.ErrExist) returns true. See [syscall.Errno.Is](https://pkg.go.dev/syscall#Errno.Is) for an example in the standard library. An Is method should only shallowly compare err and the target and not call [Unwrap](https://pkg.go.dev/errors#Unwrap) on either.\n\nExample [\u00b6](https://pkg.go.dev/errors#example-Is \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n\t\"os\"\n)\n\nfunc main() {\n\tif _, err := os.Open(\"non-existing\"); err != nil {\n\t\tif errors.Is(err, fs.ErrNotExist) {\n\t\t\tfmt.Println(\"file does not exist\")\n\t\t} else {\n\t\t\tfmt.Println(err)\n\t\t}\n\t}\n\n}\n```\n\n```\nOutput:\nfile does not exist\n```\n\nShare Format Run\n\nExample (Custom\\_match) [\u00b6](https://pkg.go.dev/errors#example-Is-Custom_match \"Go to Example (Custom_match)\")\n\nCustom errors can implement a method \"Is(error) bool\" to match other error values, overriding the default matching of [errors.Is](https://pkg.go.dev/errors#Is).\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"io/fs\"\n)\n\ntype MyIsError struct {\n\terr string\n}\n\nfunc (e MyIsError) Error() string {\n\treturn e.err\n}\nfunc (e MyIsError) Is(err error) bool {\n\treturn err == fs.ErrPermission\n}\n\nfunc main() {\n\tvar err error = MyIsError{\"an error\"}\n\tfmt.Println(\"Error equals fs.ErrPermission:\", err == fs.ErrPermission)\n\tfmt.Println(\"Error is fs.ErrPermission:\", errors.Is(err, fs.ErrPermission))\n\n}\n```\n\n```\nOutput:\nError equals fs.ErrPermission: false\nError is fs.ErrPermission: true\n```\n\nShare Format Run\n\n#### func [Join](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/join.go;l=20) [\u00b6](https://pkg.go.dev/errors#Join \"Go to Join\") added in go1.20\n\n```\nfunc Join(errs ...error) error\n```\n\nJoin returns an error that wraps the given errors. Any nil error values are discarded. Join returns nil if every value in errs is nil. The error formats as the concatenation of the strings obtained by calling the Error method of each element of errs, with a newline between each string.\n\nA non-nil error returned by Join implements the Unwrap() \\[]error method. The errors may be inspected with [Is](https://pkg.go.dev/errors#Is) and [As](https://pkg.go.dev/errors#As).\n\nExample [\u00b6](https://pkg.go.dev/errors#example-Join \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n)\n\nfunc main() {\n\terr1 := errors.New(\"err1\")\n\terr2 := errors.New(\"err2\")\n\terr := errors.Join(err1, err2)\n\tfmt.Println(err)\n\tif errors.Is(err, err1) {\n\t\tfmt.Println(\"err is err1\")\n\t}\n\tif errors.Is(err, err2) {\n\t\tfmt.Println(\"err is err2\")\n\t}\n\tfmt.Println(err.(interface{ Unwrap() []error }).Unwrap())\n}\n```\n\n```\nOutput:\nerr1\nerr2\nerr is err1\nerr is err2\n[err1 err2]\n```\n\nShare Format Run\n\n#### func [New](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/errors.go;l=64) [\u00b6](https://pkg.go.dev/errors#New \"Go to New\")\n\n```\nfunc New(text string) error\n```\n\nNew returns an error that formats as the given text. Each call to New returns a distinct error value even if the text is identical.\n\nExample [\u00b6](https://pkg.go.dev/errors#example-New \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n)\n\nfunc main() {\n\terr := errors.New(\"emit macho dwarf: elf header corrupted\")\n\tif err != nil {\n\t\tfmt.Print(err)\n\t}\n}\n```\n\n```\nOutput:\nemit macho dwarf: elf header corrupted\n```\n\nShare Format Run\n\nExample (Errorf) [\u00b6](https://pkg.go.dev/errors#example-New-Errorf \"Go to Example (Errorf)\")\n\nThe fmt package's Errorf function lets us use the package's formatting features to create descriptive error messages.\n\n```\npackage main\n\nimport (\n\t\"fmt\"\n)\n\nfunc main() {\n\tconst name, id = \"bimmler\", 17\n\terr := fmt.Errorf(\"user %q (id %d) not found\", name, id)\n\tif err != nil {\n\t\tfmt.Print(err)\n\t}\n}\n```\n\n```\nOutput:\nuser \"bimmler\" (id 17) not found\n```\n\nShare Format Run\n\nExample (Unique) [\u00b6](https://pkg.go.dev/errors#example-New-Unique \"Go to Example (Unique)\")\n\nEach call to [errors.New](https://pkg.go.dev/errors#New) returns an unique instance of the error, even if the arguments are the same. To match against errors created by [errors.New](https://pkg.go.dev/errors#New), declare a sentinel error and reuse it.\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n)\n\nfunc OopsNew() error {\n\treturn errors.New(\"an error\")\n}\n\nvar ErrSentinel = errors.New(\"an error\")\n\nfunc OopsSentinel() error {\n\treturn ErrSentinel\n}\n\nfunc main() {\n\terr1 := OopsNew()\n\terr2 := OopsNew()\n\tfmt.Println(\"Errors using distinct errors.New calls:\")\n\tfmt.Printf(\"Is(%q, %q) = %v\\n\", err1, err2, errors.Is(err1, err2))\n\n\terr3 := OopsSentinel()\n\terr4 := OopsSentinel()\n\tfmt.Println()\n\tfmt.Println(\"Errors using a sentinel error:\")\n\tfmt.Printf(\"Is(%q, %q) = %v\\n\", err3, err4, errors.Is(err3, err4))\n\n}\n```\n\n```\nOutput:\nErrors using distinct errors.New calls:\nIs(\"an error\", \"an error\") = false\n\nErrors using a sentinel error:\nIs(\"an error\", \"an error\") = true\n```\n\nShare Format Run\n\n#### func [Unwrap](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/wrap.go;l=17) [\u00b6](https://pkg.go.dev/errors#Unwrap \"Go to Unwrap\") added in go1.13\n\n```\nfunc Unwrap(err error) error\n```\n\nUnwrap returns the result of calling the Unwrap method on err, if err's type contains an Unwrap method returning error. Otherwise, Unwrap returns nil.\n\nUnwrap only calls a method of the form \"Unwrap() error\". In particular Unwrap does not unwrap errors returned by [Join](https://pkg.go.dev/errors#Join).\n\nExample [\u00b6](https://pkg.go.dev/errors#example-Unwrap \"Go to Example\")\n\n```\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n)\n\nfunc main() {\n\terr1 := errors.New(\"error1\")\n\terr2 := fmt.Errorf(\"error2: [%w]\", err1)\n\tfmt.Println(err2)\n\tfmt.Println(errors.Unwrap(err2))\n}\n```\n\n```\nOutput:\nerror2: [error1]\nerror1\n```\n\nShare Format Run\n\n### Types [\u00b6](https://pkg.go.dev/errors#pkg-types \"Go to Types\")\n\nThis section is empty.\n\n## Source Files [\u00b6](https://pkg.go.dev/errors#section-sourcefiles \"Go to Source Files\")\n\n[View all Source files](https://cs.opensource.google/go/go/+/go1.27.1:src/errors)\n\n- [errors.go](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/errors.go \"errors.go\")\n- [join.go](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/join.go \"join.go\")\n- [wrap.go](https://cs.opensource.google/go/go/+/go1.27.1:src/errors/wrap.go \"wrap.go\")" , 14"Hash" :"7d101187c4819a7778c887c98022a251885528ccf9ce51bc069f3fdb347f1182" 15 }, 16 { 17"URL" :"https://go.dev/blog/go1.13-errors" , 18"FinalURL" :"https://go.dev/blog/go1.13-errors" , 19"Title" :"Working with Errors in Go 1.13 - The Go Programming Language" , 20"Markdown" :"# [The Go Blog](https://go.dev/blog/)\n\n# Working with Errors in Go 1.13\n\nDamien Neil and Jonathan Amsterdam \n17 October 2019\n\n## Introduction\n\nGo\u2019s treatment of [errors as values](https://go.dev/blog/errors-are-values) has served us well over the last decade. Although the standard library\u2019s support for errors has been minimal\u2014just the `errors.New` and `fmt.Errorf` functions, which produce errors that contain only a message\u2014the built-in `error` interface allows Go programmers to add whatever information they desire. All it requires is a type that implements an `Error` method:\n\n```\ntype QueryError struct {\n Query string\n Err error\n}\n\nfunc (e *QueryError) Error() string { return e.Query + \": \" + e.Err.Error() }\n```\n\nError types like this one are ubiquitous, and the information they store varies widely, from timestamps to filenames to server addresses. Often, that information includes another, lower-level error to provide additional context.\n\nThe pattern of one error containing another is so pervasive in Go code that, after [extensive discussion](https://go.dev/issue/29934), Go 1.13 added explicit support for it. This post describes the additions to the standard library that provide that support: three new functions in the `errors` package, and a new formatting verb for `fmt.Errorf`.\n\nBefore describing the changes in detail, let\u2019s review how errors are examined and constructed in previous versions of the language.\n\n## Errors before Go 1.13\n\n### Examining errors\n\nGo errors are values. Programs make decisions based on those values in a few ways. The most common is to compare an error to `nil` to see if an operation failed.\n\n```\nif err != nil {\n // something went wrong\n}\n```\n\nSometimes we compare an error to a known *sentinel* value, to see if a specific error has occurred.\n\n```\nvar ErrNotFound = errors.New(\"not found\")\n\nif err == ErrNotFound {\n // something wasn't found\n}\n```\n\nAn error value may be of any type which satisfies the language-defined `error` interface. A program can use a type assertion or type switch to view an error value as a more specific type.\n\n```\ntype NotFoundError struct {\n Name string\n}\n\nfunc (e *NotFoundError) Error() string { return e.Name + \": not found\" }\n\nif e, ok := err.(*NotFoundError); ok {\n // e.Name wasn't found\n}\n```\n\n### Adding information\n\nFrequently a function passes an error up the call stack while adding information to it, like a brief description of what was happening when the error occurred. A simple way to do this is to construct a new error that includes the text of the previous one:\n\n```\nif err != nil {\n return fmt.Errorf(\"decompress %v: %v\", name, err)\n}\n```\n\nCreating a new error with `fmt.Errorf` discards everything from the original error except the text. As we saw above with `QueryError`, we may sometimes want to define a new error type that contains the underlying error, preserving it for inspection by code. Here is `QueryError` again:\n\n```\ntype QueryError struct {\n Query string\n Err error\n}\n```\n\nPrograms can look inside a `*QueryError` value to make decisions based on the underlying error. You\u2019ll sometimes see this referred to as \u201cunwrapping\u201d the error.\n\n```\nif e, ok := err.(*QueryError); ok && e.Err == ErrPermission {\n // query failed because of a permission problem\n}\n```\n\nThe `os.PathError` type in the standard library is another example of one error which contains another.\n\n## Errors in Go 1.13\n\n### The Unwrap method\n\nGo 1.13 introduces new features to the `errors` and `fmt` standard library packages to simplify working with errors that contain other errors. The most significant of these is a convention rather than a change: an error which contains another may implement an `Unwrap` method returning the underlying error. If `e1.Unwrap()` returns `e2`, then we say that `e1` *wraps* `e2`, and that you can *unwrap* `e1` to get `e2`.\n\nFollowing this convention, we can give the `QueryError` type above an `Unwrap` method that returns its contained error:\n\n```\nfunc (e *QueryError) Unwrap() error { return e.Err }\n```\n\nThe result of unwrapping an error may itself have an `Unwrap` method; we call the sequence of errors produced by repeated unwrapping the *error chain*.\n\n### Examining errors with Is and As\n\nThe Go 1.13 `errors` package includes two new functions for examining errors: `Is` and `As`.\n\nThe `errors.Is` function compares an error to a value.\n\n```\n// Similar to:\n// if err == ErrNotFound { \u2026 }\nif errors.Is(err, ErrNotFound) {\n // something wasn't found\n}\n```\n\nThe `As` function tests whether an error is a specific type.\n\n```\n// Similar to:\n// if e, ok := err.(*QueryError); ok { \u2026 }\nvar e *QueryError\n// Note: *QueryError is the type of the error.\nif errors.As(err, &e) {\n // err is a *QueryError, and e is set to the error's value\n}\n```\n\nIn the simplest case, the `errors.Is` function behaves like a comparison to a sentinel error, and the `errors.As` function behaves like a type assertion. When operating on wrapped errors, however, these functions consider all the errors in a chain. Let\u2019s look again at the example from above of unwrapping a `QueryError` to examine the underlying error:\n\n```\nif e, ok := err.(*QueryError); ok && e.Err == ErrPermission {\n // query failed because of a permission problem\n}\n```\n\nUsing the `errors.Is` function, we can write this as:\n\n```\nif errors.Is(err, ErrPermission) {\n // err, or some error that it wraps, is a permission problem\n}\n```\n\nThe `errors` package also includes a new `Unwrap` function which returns the result of calling an error\u2019s `Unwrap` method, or `nil` when the error has no `Unwrap` method. It is usually better to use `errors.Is` or `errors.As`, however, since these functions will examine the entire chain in a single call.\n\nNote: although it may feel odd to take a pointer to a pointer, in this case it is correct. Think of it instead as taking a pointer to a value of the error type; it so happens in this case that the returned error is a pointer type.\n\n### Wrapping errors with %w\n\nAs mentioned earlier, it is common to use the `fmt.Errorf` function to add additional information to an error.\n\n```\nif err != nil {\n return fmt.Errorf(\"decompress %v: %v\", name, err)\n}\n```\n\nIn Go 1.13, the `fmt.Errorf` function supports a new `%w` verb. When this verb is present, the error returned by `fmt.Errorf` will have an `Unwrap` method returning the argument of `%w`, which must be an error. In all other ways, `%w` is identical to `%v`.\n\n```\nif err != nil {\n // Return an error which unwraps to err.\n return fmt.Errorf(\"decompress %v: %w\", name, err)\n}\n```\n\nWrapping an error with `%w` makes it available to `errors.Is` and `errors.As`:\n\n```\nerr := fmt.Errorf(\"access denied: %w\", ErrPermission)\n...\nif errors.Is(err, ErrPermission) ...\n```\n\n### Whether to Wrap\n\nWhen adding additional context to an error, either with `fmt.Errorf` or by implementing a custom type, you need to decide whether the new error should wrap the original. There is no single answer to this question; it depends on the context in which the new error is created. Wrap an error to expose it to callers. Do not wrap an error when doing so would expose implementation details.\n\nAs one example, imagine a `Parse` function which reads a complex data structure from an `io.Reader`. If an error occurs, we wish to report the line and column number at which it occurred. If the error occurs while reading from the `io.Reader`, we will want to wrap that error to allow inspection of the underlying problem. Since the caller provided the `io.Reader` to the function, it makes sense to expose the error produced by it.\n\nIn contrast, a function which makes several calls to a database probably should not return an error which unwraps to the result of one of those calls. If the database used by the function is an implementation detail, then exposing these errors is a violation of abstraction. For example, if the `LookupUser` function of your package `pkg` uses Go\u2019s `database/sql` package, then it may encounter a `sql.ErrNoRows` error. If you return that error with `fmt.Errorf(\"accessing DB: %v\", err)` then a caller cannot look inside to find the `sql.ErrNoRows`. But if the function instead returns `fmt.Errorf(\"accessing DB: %w\", err)`, then a caller could reasonably write\n\n```\nerr := pkg.LookupUser(...)\nif errors.Is(err, sql.ErrNoRows) \u2026\n```\n\nAt that point, the function must always return `sql.ErrNoRows` if you don\u2019t want to break your clients, even if you switch to a different database package. In other words, wrapping an error makes that error part of your API. If you don\u2019t want to commit to supporting that error as part of your API in the future, you shouldn\u2019t wrap the error.\n\nIt\u2019s important to remember that whether you wrap or not, the error text will be the same. A *person* trying to understand the error will have the same information either way; the choice to wrap is about whether to give *programs* additional information so they can make more informed decisions, or to withhold that information to preserve an abstraction layer.\n\n## Customizing error tests with Is and As methods\n\nThe `errors.Is` function examines each error in a chain for a match with a target value. By default, an error matches the target if the two are [equal](https://go.dev/ref/spec#Comparison_operators). In addition, an error in the chain may declare that it matches a target by implementing an `Is` *method*.\n\nAs an example, consider this error inspired by the [Upspin error package](https://commandcenter.blogspot.com/2017/12/error-handling-in-upspin.html) which compares an error against a template, considering only fields which are non-zero in the template:\n\n```\ntype Error struct {\n Path string\n User string\n}\n\nfunc (e *Error) Is(target error) bool {\n t, ok := target.(*Error)\n if !ok {\n return false\n }\n return (e.Path == t.Path || t.Path == \"\") &&\n (e.User == t.User || t.User == \"\")\n}\n\nif errors.Is(err, &Error{User: \"someuser\"}) {\n // err's User field is \"someuser\".\n}\n```\n\nThe `errors.As` function similarly consults an `As` method when present.\n\n## Errors and package APIs\n\nA package which returns errors (and most do) should describe what properties of those errors programmers may rely on. A well-designed package will also avoid returning errors with properties that should not be relied upon.\n\nThe simplest specification is to say that operations either succeed or fail, returning a nil or non-nil error value respectively. In many cases, no further information is needed.\n\nIf we wish a function to return an identifiable error condition, such as \u201citem not found,\u201d we might return an error wrapping a sentinel.\n\n```\nvar ErrNotFound = errors.New(\"not found\")\n\n// FetchItem returns the named item.\n//\n// If no item with the name exists, FetchItem returns an error\n// wrapping ErrNotFound.\nfunc FetchItem(name string) (*Item, error) {\n if itemNotFound(name) {\n return nil, fmt.Errorf(\"%q: %w\", name, ErrNotFound)\n }\n // ...\n}\n```\n\nThere are other existing patterns for providing errors which can be semantically examined by the caller, such as directly returning a sentinel value, a specific type, or a value which can be examined with a predicate function.\n\nIn all cases, care should be taken not to expose internal details to the user. As we touched on in \u201cWhether to Wrap\u201d above, when you return an error from another package you should convert the error to a form that does not expose the underlying error, unless you are willing to commit to returning that specific error in the future.\n\n```\nf, err := os.Open(filename)\nif err != nil {\n // The *os.PathError returned by os.Open is an internal detail.\n // To avoid exposing it to the caller, repackage it as a new\n // error with the same text. We use the %v formatting verb, since\n // %w would permit the caller to unwrap the original *os.PathError.\n return fmt.Errorf(\"%v\", err)\n}\n```\n\nIf a function is defined as returning an error wrapping some sentinel or type, do not return the underlying error directly.\n\n```\nvar ErrPermission = errors.New(\"permission denied\")\n\n// DoSomething returns an error wrapping ErrPermission if the user\n// does not have permission to do something.\nfunc DoSomething() error {\n if !userHasPermission() {\n // If we return ErrPermission directly, callers might come\n // to depend on the exact error value, writing code like this:\n //\n // if err := pkg.DoSomething(); err == pkg.ErrPermission { \u2026 }\n //\n // This will cause problems if we want to add additional\n // context to the error in the future. To avoid this, we\n // return an error wrapping the sentinel so that users must\n // always unwrap it:\n //\n // if err := pkg.DoSomething(); errors.Is(err, pkg.ErrPermission) { ... }\n return fmt.Errorf(\"%w\", ErrPermission)\n }\n // ...\n}\n```\n\n## Conclusion\n\nAlthough the changes we\u2019ve discussed amount to just three functions and a formatting verb, we hope they will go a long way toward improving how errors are handled in Go programs. We expect that wrapping to provide additional context will become commonplace, helping programs to make better decisions and helping programmers to find bugs more quickly.\n\nAs Russ Cox said in his [GopherCon 2019 keynote](https://go.dev/blog/experiment), on the path to Go 2 we experiment, simplify and ship. Now that we\u2019ve shipped these changes, we look forward to the experiments that will follow.\n\n**Next article:** [Go Modules: v2 and Beyond](https://go.dev/blog/v2-go-modules) \n**Previous article:** [Publishing Go Modules](https://go.dev/blog/publishing-go-modules) \n[**Blog Index**](https://go.dev/blog/all)" , 21"Hash" :"376defe0a35396dcd7f31c2ca3a78944a6be909b9bfb50485ca0f6fb1c0791d5" 22 } 23]