|
2024-07-20
| ||
| 23:44 | Fix [54329e39c7]. check-in: 7510b22367 user: pooryorick tags: unchained, INCOMPATIBLE_LICENSE | |
|
2023-07-12
| ||
| 07:40 | • Ticket [54329e39c7] Compiled foreach uses excessive memory for lseq sequences status still Closed with 6 other changes artifact: bb35b27107 user: jan.nijtmans | |
| 06:44 | • Closed ticket [54329e39c7]. artifact: b9cd02b2b4 user: apnadkarni | |
| 06:39 | Fix [54329e39c7] check-in: 95cab70fe8 user: apnadkarni tags: core-8-branch | |
| 02:51 | Fix [54329e39c7] check-in: 3435cde0f5 user: apnadkarni tags: trunk, main | |
|
2023-07-11
| ||
| 16:41 | • Ticket [54329e39c7] Compiled foreach uses excessive memory for lseq sequences status still Open with 3 other changes artifact: 07b194893b user: apnadkarni | |
| 16:17 | • Ticket [54329e39c7]: 3 changes artifact: 75b6e3a304 user: apnadkarni | |
| 15:34 | • Ticket [54329e39c7]: 3 changes artifact: 5e5a2c7c27 user: griffin | |
| 12:14 | • Ticket [54329e39c7]: 3 changes artifact: b67cadd69b user: jan.nijtmans | |
| 11:36 | • Ticket [54329e39c7]: 3 changes artifact: 962f83d9d4 user: apnadkarni | |
| 10:55 | • New ticket [54329e39c7]. artifact: a9efa64d2c user: apnadkarni | |
| Ticket UUID: | 54329e39c757ee2aac298a142ecfe2a7ebe72391 | |||
| Title: | Compiled foreach uses excessive memory for lseq sequences | |||
| Type: | Bug | Version: | 9.0 | |
| Submitter: | apnadkarni | Created on: | 2023-07-11 10:55:35 | |
| Subsystem: | - New Builtin Commands | Assigned To: | apnadkarni | |
| Priority: | 5 Medium | Severity: | Minor | |
| Status: | Closed | Last Modified: | 2023-07-12 07:40:09 | |
| Resolution: | Fixed | Closed By: | jan.nijtmans | |
| Closed on: | 2023-07-12 07:40:09 | |||
| Description: |
This is a memory usage issue, not bug. As shown below, a foreach loop over a lseq within a proc results in a loss of the memory efficiency benefit of lseq.
```
% set l [lseq 1000000]; llength $l
1000000
% proc getmem {} {set fd [open /proc/[pid]/statm]; return [read $fd][close $fd]}
% getmem
6026 1777 1468 558 0 412 0
% foreach x $l {}
% getmem
6026 1777 1468 558 0 412 0
% proc p l {foreach x $l {}}
% p $l
% getmem
17731 13556 1477 558 0 12117 0
```
Notice the jump in memory usage after the loop in the proc. The same loop on the command line does not increase memory.
I suspect the cause is the use of Tcl_ListObjGetElements somewhere in the execution engine in lieu of Tcl_ListObjIndex.
| |||
| User Comments: |
jan.nijtmans added on 2023-07-12 07:40:09:
Many thanks, Ashok! apnadkarni added on 2023-07-12 06:44:56: Fixed for [9.0](https://core.tcl-lang.org/tcl/info/3435cde0f55776b4) and [8.7](https://core.tcl-lang.org/tcl/info/95cab70fe8b80e1c). apnadkarni added on 2023-07-11 16:41:07: Was in the process of doing that distinction. There is a some measurable difference for "traditional" lists between the getelements and index. A little surprising to me because the getelements there happens on every iteration just like the index. On the other hand, the former is a macro and the latter a function so that might be something I look at later. apnadkarni added on 2023-07-11 16:17:15: I already had inserted the CACHE macros and was coding the change to distinguish between abstract and non-abstract probably as you were updating the ticket :-) The original is more efficient for non-abstract lists though I am a bit surprised at the difference. Code-wise there is very little difference calling Tcl_ListObjIndex noting that the GetElements in that fragment is done on *every* iteration. But its measurable, may be some cache effects? Who knows. Still doing performance measurements. /Ashok griffin added on 2023-07-11 15:34:44: The proposed fix looks good. I wonder if it is worthwhile to have separate loops for Lists vs Abstract Lists, as GetElements is more efficient for Lists. It may be that the difference is not worth the complexity. IDK. Note that there are a number of places where DECACHE/CACHE calls need to be inserted in TBCE. I've done that in tip-636 (main). [tclExecute.c](https://core.tcl-lang.org/tcl/fdiff?v1=b7ee7c30a6f02625&v2=5d734646930c5089) jan.nijtmans added on 2023-07-11 12:14:16: Just one remark: I guess Tcl 8.7 has the same problem. So I recommend backporting this (which can be done after merging this to trunk) +1 apnadkarni added on 2023-07-11 11:36:54: Proposed fix in branch bug-54329e39c7. Needs review. | |||