html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1793274350,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1793274350,IC_kwDOCGYnMM5q4zHu,9599,2023-11-04T00:46:30Z,2023-11-04T00:46:30Z,OWNER,"And a GIF of the fix after applying:
- #598
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1793273968,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1793273968,IC_kwDOCGYnMM5q4zBw,9599,2023-11-04T00:45:19Z,2023-11-04T00:45:19Z,OWNER,"Here's an animated GIF that demonstrates the bug:
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1747231893,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1747231893,IC_kwDOCGYnMM5oJKSV,62745,2023-10-04T16:15:09Z,2023-10-04T16:28:21Z,CONTRIBUTOR,"I confirm the bug, as above, and that @jonafato 's patch fixes it for me. However, it's not the right fix. The problem is that ProgressBar is being used in the wrong way. This also results in two lines being printed instead of one, like this:
```
[#######-----------------------------] 20%
[####################################] 100%%
```
The bug is reproducible for me in any terminal, including Gnome Terminal and Guake, and VSCode. With VSCode I can use this launch.json to reproduce it:
```json
{
""version"": ""0.2.0"",
""configurations"": [
{
""name"": ""Python: Module"",
""type"": ""python"",
""request"": ""launch"",
""module"": ""sqlite_utils"",
""justMyCode"": false,
""args"": [""insert"", ""test.db"", ""test"", ""--csv"", ""tests/sniff/example1.csv""]
}
]
}
```
[edit - deleted my analysis of why the current code is wrong, which was confused and confusing]","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1640826795,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1640826795,IC_kwDOCGYnMM5hzQer,76528036,2023-07-18T19:08:50Z,2023-07-18T19:08:50Z,NONE,"Came here to report this, but instead I'll confirm the issue across two terminal emulators (Gnome Terminal and Alacritty) on Pop_OS! 22.04 (currently based on Ubuntu/Gnome). Also messes up the formatting of the terminal. Can also confirm that reset fixes it until the next sqlite-utils command. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1578840450,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1578840450,IC_kwDOCGYnMM5eGzGC,392720,2023-06-06T14:09:04Z,2023-06-06T14:09:04Z,NONE,"I also ran into this recently. See below for a patch for one possible solution (tested via ""it works on my machine"", but I don't expect that this behavior would vary a whole lot across terminal emulators and shells). Another possible solution might be to subclass click's `ProgressBar` to keep the logic within the original context manager. Happy to send a PR or for this patch to serve as the basis for a fix that someone else authors.
```patch
diff --git a/sqlite_utils/utils.py b/sqlite_utils/utils.py
index 06c1a4c..530a3a3 100644
--- a/sqlite_utils/utils.py
+++ b/sqlite_utils/utils.py
@@ -147,14 +147,23 @@ def decode_base64_values(doc):
class UpdateWrapper:
- def __init__(self, wrapped, update):
+ def __init__(self, wrapped, update, render_finish):
self._wrapped = wrapped
self._update = update
+ self._render_finish = render_finish
def __iter__(self):
- for line in self._wrapped:
- self._update(len(line))
- yield line
+ return self
+
+ def __next__(self):
+ try:
+ line = next(self._wrapped)
+ except StopIteration as e:
+ self._render_finish()
+ raise
+
+ self._update(len(line))
+ return line
def read(self, size=-1):
data = self._wrapped.read(size)
@@ -178,7 +187,7 @@ def file_progress(file, silent=False, **kwargs):
else:
file_length = os.path.getsize(file.name)
with click.progressbar(length=file_length, **kwargs) as bar:
- yield UpdateWrapper(file, bar.update)
+ yield UpdateWrapper(file, bar.update, bar.render_finish)
class Format(enum.Enum):
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1444474487,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1444474487,IC_kwDOCGYnMM5WGO53,167893,2023-02-24T20:57:43Z,2023-02-24T22:22:18Z,CONTRIBUTOR,"I think I see what is happening here, although I haven't quite work out a fix yet. Usually:
* `click.progressbar.render_progress()` renders the cursor invisible on each invocation (update of the bar)
* When the progress bar goes out of scope, the `__exit()__` method is invoked, which calls `render_finish()` to make the cursor re-appear.
(See terminal escape sequences `BEFORE_BAR` and `AFTER_BAR` in click).
However the sqlite-utils `utils.file_progress` context manager wraps `click.progressbar` and yields an instance of a helper class:
``` python
@contextlib.contextmanager
def file_progress(file, silent=False, **kwargs):
...
with click.progressbar(length=file_length, **kwargs) as bar:
yield UpdateWrapper(file, bar.update)
```
The yielded `UpdateWrapper` goes out of scope quickly and `click.progressbar.__exit__()` is called. The cursor is made un-invisible. Hoewever `bar` is still live and so when the caller iterates on the yielded wrapper this invokes the bar's update method, calling `render_progress()`, each time printing the ""make cursor invisible"" escape code. The `progressbar.__exit__` function is not called again, so the cursor doesn't re-appear.
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1416486796,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1416486796,IC_kwDOCGYnMM5Ubd-M,16236421,2023-02-03T22:32:10Z,2023-02-03T22:32:10Z,NONE,Came here to say that I also have this issue.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1252898131,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1252898131,IC_kwDOCGYnMM5KrbVT,7908073,2022-09-20T20:51:21Z,2022-09-20T20:56:07Z,CONTRIBUTOR,"When I run `reset` it fixes my terminal. I suspect it is related to the progress bar
https://linux.die.net/man/1/reset
```
950 1s /m/d/03_Downloads 🐑 echo $TERM
xterm-kitty
▓░▒░ /m/d/03_Downloads 🌏 kitty -v
kitty 0.26.2 created by Kovid Goyal
$ sqlite-utils insert test.db facility facility-boundary-us-all.csv --csv
blah blah blah (no offense)
$
$ reset
$
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1155749696,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1155749696,IC_kwDOCGYnMM5E41dA,9599,2022-06-14T21:57:05Z,2022-06-14T21:57:05Z,OWNER,Marking this as help wanted because I can't figure out how to replicate it!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,
https://github.com/simonw/sqlite-utils/issues/433#issuecomment-1139484453,https://api.github.com/repos/simonw/sqlite-utils/issues/433,1139484453,IC_kwDOCGYnMM5D6ycl,4068,2022-05-27T10:20:08Z,2022-05-27T10:20:08Z,NONE,I can confirm. This only happens with sqlite-utils. I am using gnome-terminal with bash.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1239034903,