gnuplot-beta Mailing List for gnuplot (Page 34)

----- Original Message -----

>>  In file included from ../../gnuplot-gnuplot-main/src/term.h:227:0,
>>                   from ../../gnuplot-gnuplot-main/src/term.c:1189:
>>  ../../gnuplot-gnuplot-main/term/caca.trm: In function 
> 'CACA_init_display':
>>  ../../gnuplot-gnuplot-main/term/caca.trm:771:23: error: 
> 'current_locale' undeclared (first use in this function); did you mean 
> 'surface_lscale'?
>>      setlocale(LC_TIME, current_locale != NULL ? current_locale : 
> "C");
>>                         ^~~~~~~~~~~~~~
>>                         surface_lscale
>>  ../../gnuplot-gnuplot-main/term/caca.trm:771:23: note: each undeclared 
> identifier is reported only once for each function it appears in
>>  make[4]: *** [Makefile:910: term.o] Error 1
>> 
>> 
>>  Tatsuro
> 
> My bad.  I missed that one.
> I am starting to think that all this locale fixup should be done by the 
> "set term"
> command rather than by individual terminal drivers.
> 
>     Ethan
> 


Thanks for quick fix.

> I am starting to think that all this locale fixup should be done by the 
> "set term"
> command rather than by individual terminal drivers.

It's a nice change. 

Tatsuro

On Thursday, September 6, 2018 4:11:53 PM PDT Tatsuro MATSUOKA wrote:
> In file included from ../../gnuplot-gnuplot-main/src/term.h:227:0,
>                  from ../../gnuplot-gnuplot-main/src/term.c:1189:
> ../../gnuplot-gnuplot-main/term/caca.trm: In function 'CACA_init_display':
> ../../gnuplot-gnuplot-main/term/caca.trm:771:23: error: 'current_locale' undeclared (first use in this function); did you mean 'surface_lscale'?
>     setlocale(LC_TIME, current_locale != NULL ? current_locale : "C");
>                        ^~~~~~~~~~~~~~
>                        surface_lscale
> ../../gnuplot-gnuplot-main/term/caca.trm:771:23: note: each undeclared identifier is reported only once for each function it appears in
> make[4]: *** [Makefile:910: term.o] Error 1
> 
> 
> Tatsuro

My bad.  I missed that one.
I am starting to think that all this locale fixup should be done by the "set term"
command rather than by individual terminal drivers.

	Ethan

In file included from ../../gnuplot-gnuplot-main/src/term.h:227:0,
                 from ../../gnuplot-gnuplot-main/src/term.c:1189:
../../gnuplot-gnuplot-main/term/caca.trm: In function 'CACA_init_display':
../../gnuplot-gnuplot-main/term/caca.trm:771:23: error: 'current_locale' undeclared (first use in this function); did you mean 'surface_lscale'?
    setlocale(LC_TIME, current_locale != NULL ? current_locale : "C");
                       ^~~~~~~~~~~~~~
                       surface_lscale
../../gnuplot-gnuplot-main/term/caca.trm:771:23: note: each undeclared identifier is reported only once for each function it appears in
make[4]: *** [Makefile:910: term.o] Error 1


Tatsuro

On Thursday, September 6, 2018 3:28:24 PM PDT Tatsuro MATSUOKA wrote:
> In development source,
> commit [8fc9b3]
> 
> By default do not use multithreaded wxtgtk 
> 
> I have been using --with-wx-single-threaded for Unixy build for a long time.
> 
> Is there any plan that this commit will be back-ported to stable source?

Yes.

I have not caught up with back-porting recent changes to 5.3 for inclusion in
the stable branch also, but this change is definitely on the list.

I can no longer build a working wxt terminal for linux with multithreading
enabled, and that is true for both the stable and development versions.

	Ethan

In development source,
commit [8fc9b3]

By default do not use multithreaded wxtgtk 

I have been using --with-wx-single-threaded for Unixy build for a long time.

Is there any plan that this commit will be back-ported to stable source?

Tatsuro

On Sunday, 19 August 2018 23:52:05 Dima Kogan wrote:
> sfeam <sf...@us...> writes:
> 
> > On Sunday, 19 August 2018 20:25:24 Dima Kogan wrote:
> >> Dima Kogan <gn...@di...> writes:
> >>
> >> > Ethan A Merritt <sf...@us...> writes:
> >> >
> >> >> On Friday, August 17, 2018 1:15:24 PM PDT Dima Kogan wrote:
> >> >>>
> >> >>> I'm looking for a feature that (I think) doesn't exist currently. Want
> >> >>> to ask first, in case something DOES exist that I'm not seeing.
> >> >>
> >> >> The command you are looking for is
> >> >> 	pause mouse close
> >>
> >> This works in its most basic form, which is awesome. Thanks! But I'm now
> >> going to be a pain in the butt. For a gnuplot-using application to work
> >> reasonably, this needs to only wait for a window-close event when one is
> >> coming: i.e. when a window is open. Scenarios that could happen:
> >>
> >> 1. User already closed the window before I send my "pause mouse close"
> >
> > That sounds like a case of "well then don't do that".
> > If the parent program sends the plot command and the pause command
> > in immediate succession the time window in which that race could happen
> > would be very small.  If the commands do not come in immediate
> > succession then maybe the program logic needs re-thinking.
> > Can you give a more complete example of the flow of events?
> >
> >> 2. We're using a non-interactive terminal (dumb, pdf, etc etc)
> >> Ideas on handling that?
> >
> > So far as I know, in that case the next <\n> in the input stream
> > will terminal the "pause" command.  Does that not work for you?
> 
> I played with it a bit more just now, and it's good-enough. It would be
> NICE if "pause mouse close" could know if there's anything to
> potentially wait for, but I guess I don't strictly need it.

If you have identified a sequence of reasonable events that causes gnuplot
to lock up entirely, I would consider that a bug.  In the case of 
"pause mouse" I agree that if you issue this command in the state that
the current terminal is mouseable but the most recent plot window has
been closed then you have a potential lock-up.  

I say "potential" because from a normal terminal session you can break the
lock by typing Ctrl-C.  This resets error and pause conditions and returns
you to the gnuplot command line.   

However, if gnuplot is being run from some non-terminal controlling process
I am not entirely clear on who would have to send SIGINT (Ctrl-C) to whom.
I don't understand you usage scenario well enough to guess.
Does the user who is operating the mouse also have a terminal window open?
If Ctrl-C is typed in that window, what process receives the signal?

	Ethan
  

> Thanks much. This will fix a long-standing annoyance with gnuplotlib

sfeam <sf...@us...> writes:

> On Sunday, 19 August 2018 20:25:24 Dima Kogan wrote:
>> Dima Kogan <gn...@di...> writes:
>>
>> > Ethan A Merritt <sf...@us...> writes:
>> >
>> >> On Friday, August 17, 2018 1:15:24 PM PDT Dima Kogan wrote:
>> >>>
>> >>> I'm looking for a feature that (I think) doesn't exist currently. Want
>> >>> to ask first, in case something DOES exist that I'm not seeing.
>> >>
>> >> The command you are looking for is
>> >> 	pause mouse close
>>
>> This works in its most basic form, which is awesome. Thanks! But I'm now
>> going to be a pain in the butt. For a gnuplot-using application to work
>> reasonably, this needs to only wait for a window-close event when one is
>> coming: i.e. when a window is open. Scenarios that could happen:
>>
>> 1. User already closed the window before I send my "pause mouse close"
>
> That sounds like a case of "well then don't do that".
> If the parent program sends the plot command and the pause command
> in immediate succession the time window in which that race could happen
> would be very small.  If the commands do not come in immediate
> succession then maybe the program logic needs re-thinking.
> Can you give a more complete example of the flow of events?
>
>> 2. We're using a non-interactive terminal (dumb, pdf, etc etc)
>> Ideas on handling that?
>
> So far as I know, in that case the next <\n> in the input stream
> will terminal the "pause" command.  Does that not work for you?

I played with it a bit more just now, and it's good-enough. It would be
NICE if "pause mouse close" could know if there's anything to
potentially wait for, but I guess I don't strictly need it.

Thanks much. This will fix a long-standing annoyance with gnuplotlib

On Sunday, 19 August 2018 20:25:24 Dima Kogan wrote:
> Dima Kogan <gn...@di...> writes:
> 
> > Ethan A Merritt <sf...@us...> writes:
> >
> >> On Friday, August 17, 2018 1:15:24 PM PDT Dima Kogan wrote:
> >>>
> >>> I'm looking for a feature that (I think) doesn't exist currently. Want
> >>> to ask first, in case something DOES exist that I'm not seeing.
> >>
> >> The command you are looking for is
> >> 	pause mouse close
> 
> This works in its most basic form, which is awesome. Thanks! But I'm now
> going to be a pain in the butt. For a gnuplot-using application to work
> reasonably, this needs to only wait for a window-close event when one is
> coming: i.e. when a window is open. Scenarios that could happen:
> 
> 1. User already closed the window before I send my "pause mouse close"

That sounds like a case of "well then don't do that".
If the parent program sends the plot command and the pause command
in immediate succession the time window in which that race could happen
would be very small.  If the commands do not come in immediate
succession then maybe the program logic needs re-thinking.
Can you give a more complete example of the flow of events?

> 2. We're using a non-interactive terminal (dumb, pdf, etc etc)
> Ideas on handling that?

So far as I know, in that case the next <\n> in the input stream
will terminal the "pause" command.  Does that not work for you?

	Ethan

Dima Kogan <gn...@di...> writes:

> Ethan A Merritt <sf...@us...> writes:
>
>> On Friday, August 17, 2018 1:15:24 PM PDT Dima Kogan wrote:
>>>
>>> I'm looking for a feature that (I think) doesn't exist currently. Want
>>> to ask first, in case something DOES exist that I'm not seeing.
>>
>> The command you are looking for is
>> 	pause mouse close

This works in its most basic form, which is awesome. Thanks! But I'm now
going to be a pain in the butt. For a gnuplot-using application to work
reasonably, this needs to only wait for a window-close event when one is
coming: i.e. when a window is open. Scenarios that could happen:

1. User already closed the window before I send my "pause mouse close"

2. We're using a non-interactive terminal (dumb, pdf, etc etc)

Ideas on handling that?

Ethan A Merritt <sf...@us...> writes:

> On Friday, August 17, 2018 1:15:24 PM PDT Dima Kogan wrote:
>>
>> I'm looking for a feature that (I think) doesn't exist currently. Want
>> to ask first, in case something DOES exist that I'm not seeing.
>
> The command you are looking for is
> 	pause mouse close
>
> Recent discussion has indicated that on some systems in may also be
> necessary add a bind command
> 	bind "Close" "exit gnuplot"
> 	pause mouse close
>
> gnuplot will exit when the user closes the plot window.
>
> If you want gnuplot to signal the parent application via some side channel
> rather rather than exiting , that's a bit trickier.
> There are several options but the details are OS dependent.
> For example (not necessary the best option)
>
> 	bind "Close" "system('kill -SIGCONT $PARENTID')"

Ooh, that looks like what I want. Thanks! Will try it out at some point.
Daniel: --persist isn't quite what I want. I'd like gnuplot to stay
alive, and to tell me when the client window has gone away.

For the record, it's for this: https://github.com/dkogan/gnuplotlib

On 08/17/2018 03:15 PM, Dima Kogan wrote:
> Hi.
> 
> I'm looking for a feature that (I think) doesn't exist currently. Want
> to ask first, in case something DOES exist that I'm not seeing.
> 
> A number of applications that make plots use gnuplot as a backend. The
> sequence you want there is:
> 
> 1. User says "make a plot"
> 
> 2. Application tells gnuplot to make a plot
> 
> 3. gnuplot does its thing using an interactive terminal (x11, qt, wxt,
>     ...) and a plot window pops up
> 
> 4. user looks at the plot, then closes the window when they're done, and
>     wants to continue using the application
> 
> The issue here is that currently the application (parent of the
> "gnuplot" process) has no way of knowing when the user finished looking
> at the plot, which necessitates ugly workarounds. For instance with
> gnuplotlib I can write a script that just makes a plot. If I just call
> gnuplotlib.plot(), then the application will make a plot and then exit
> immediately. As a workaround I put a sleep(10000) after the plot() call.
> I'd like to have some sort of gnuplotlib.wait() that blocks until
> gnuplot is done. Maybe gnuplot can be asked to print something on the
> console when the client window is closed, or something? Thoughts?

Does "gnuplot --persist" work for your needs?  In the following, the 
gnuplot_qt process hangs around until the window is closed:

@linux ~ $ gnuplot --persist

	G N U P L O T
	Version 5.3 patchlevel 0    last modified 2018-04-26

	Copyright (C) 1986-1993, 1998, 2004, 2007-2018
	Thomas Williams, Colin Kelley and many others

	gnuplot home:     http://www.gnuplot.info
	mailing list:     gnu...@li...
	faq, bugs, etc:   type "help FAQ"
	immediate help:   type "help"  (plot window: hit 'h')

Terminal type is now 'qt'
Options are '0 font "Sans,9"'
gnuplot> system "ps -e | grep gnuplot"
  6680 pts/1    00:00:00 gnuplot
gnuplot> plot 1/x
gnuplot> system "ps -e | grep gnuplot"
  6680 pts/1    00:00:00 gnuplot
  6690 ?        00:00:00 gnuplot_qt
gnuplot> exit
@linux ~ $ ps -e | grep gnuplot
  6690 ?        00:00:00 gnuplot_qt
@linux ~ $ (manually close window)
manually: command not found
@linux ~ $ ps -e | grep gnuplot
@linux ~ $

On Friday, August 17, 2018 1:15:24 PM PDT Dima Kogan wrote:
> Hi.
> 
> I'm looking for a feature that (I think) doesn't exist currently. Want
> to ask first, in case something DOES exist that I'm not seeing.

This ought to be a FAQ or something.

The command you are looking for is
	pause mouse close

Recent discussion has indicated that on some systems in may also be
necessary add a bind command
	bind "Close" "exit gnuplot"
	pause mouse close

gnuplot will exit when the user closes the plot window.

If you want gnuplot to signal the parent application via some side channel
rather rather than exiting , that's a bit trickier.
There are several options but the details are OS dependent.
For example (not necessary the best option)

	bind "Close" "system('kill -SIGCONT $PARENTID')"

	Ethan

> A number of applications that make plots use gnuplot as a backend. The
> sequence you want there is:
> 
> 1. User says "make a plot"
> 
> 2. Application tells gnuplot to make a plot
> 
> 3. gnuplot does its thing using an interactive terminal (x11, qt, wxt,
>    ...) and a plot window pops up
> 
> 4. user looks at the plot, then closes the window when they're done, and
>    wants to continue using the application
> 
> The issue here is that currently the application (parent of the
> "gnuplot" process) has no way of knowing when the user finished looking
> at the plot, which necessitates ugly workarounds. For instance with
> gnuplotlib I can write a script that just makes a plot. If I just call
> gnuplotlib.plot(), then the application will make a plot and then exit
> immediately. As a workaround I put a sleep(10000) after the plot() call.
> I'd like to have some sort of gnuplotlib.wait() that blocks until
> gnuplot is done. Maybe gnuplot can be asked to print something on the
> console when the client window is closed, or something? Thoughts?

Hi.

I'm looking for a feature that (I think) doesn't exist currently. Want
to ask first, in case something DOES exist that I'm not seeing.

A number of applications that make plots use gnuplot as a backend. The
sequence you want there is:

1. User says "make a plot"

2. Application tells gnuplot to make a plot

3. gnuplot does its thing using an interactive terminal (x11, qt, wxt,
   ...) and a plot window pops up

4. user looks at the plot, then closes the window when they're done, and
   wants to continue using the application

The issue here is that currently the application (parent of the
"gnuplot" process) has no way of knowing when the user finished looking
at the plot, which necessitates ugly workarounds. For instance with
gnuplotlib I can write a script that just makes a plot. If I just call
gnuplotlib.plot(), then the application will make a plot and then exit
immediately. As a workaround I put a sleep(10000) after the plot() call.
I'd like to have some sort of gnuplotlib.wait() that blocks until
gnuplot is done. Maybe gnuplot can be asked to print something on the
console when the client window is closed, or something? Thoughts?

It seems there's a small inaccuracy in the current (5.2.4) help for 
"missing". Here's the relevant part of the text:

<quote>
Gnuplot makes a distinction between missing data and invalid data 
(e.g. "NaN", 1/0.).  For example invalid data causes a gap in a line 
drawn through sequential data points; missing data does not.

Non-numeric characters found in a numeric field will usually be 
interpreted as invalid rather than as a missing data point unless they 
happen to match the `missing` string.
</quote>

In fact it seems that in a data block anything other than "NaN" (on a 
case-insensitive comparison) is treated as "missing" (in the 
operational sense that it does not produce a gap in a line plot), not 
invalid. Here's a little test script:

set term pngcairo
set output 'test.png'
set nokey
plot '-' using 1:2 w lines
2000 1
2001 2
2002 TEST_HERE
2003 2
2004 3
e

The plot is drawn with a line from (2001,2) to 2003,2), and that 
remains the case if I put "?", ".", "NA" or "foo" in place of 
"TEST_HERE". A gap appears only if I substitute "nan" or equivalent.

There's a practical consequence: if I'm in the habit of writing "?" 
for missing values in gnuplot input data, the doc would give me the 
idea that I could toggle "gaps in lines" behavior by inserting or
removing

set datafile missing "?"

With this set I should get no gaps, without it I should get gaps.
But get no gaps regardless.

-- 
Allin Cottrell
Department of Economics
Wake Forest University, NC

Sorry, just noticed something. The statement in the doc, "Non-numeric 
characters found in a numeric field will usually be interpreted as 
invalid rather than as a missing data point unless they happen to 
match the `missing` string" is borne out if I substitute

   plot '-' using 1:($2) w lines

for

   plot '-' using 1:2 w lines

The doc further states, "Old gnuplot versions handled NaN differently 
depending of the form of the `using` clause", but it doesn't mention 
that the form of the `using` clause matters currently too.

Allin Cottrell

On Sun, 12 Aug 2018, Allin Cottrell wrote:

> It seems there's a small inaccuracy in the current (5.2.4) help for 
> "missing". Here's the relevant part of the text:
>
> <quote>
> Gnuplot makes a distinction between missing data and invalid data (e.g. 
> "NaN", 1/0.).  For example invalid data causes a gap in a line drawn through 
> sequential data points; missing data does not.
>
> Non-numeric characters found in a numeric field will usually be interpreted 
> as invalid rather than as a missing data point unless they happen to match 
> the `missing` string.
> </quote>
>
> In fact it seems that in a data block anything other than "NaN" (on a 
> case-insensitive comparison) is treated as "missing" (in the operational 
> sense that it does not produce a gap in a line plot), not invalid. Here's a 
> little test script:
>
> set term pngcairo
> set output 'test.png'
> set nokey
> plot '-' using 1:2 w lines
> 2000 1
> 2001 2
> 2002 TEST_HERE
> 2003 2
> 2004 3
> e
>
> The plot is drawn with a line from (2001,2) to 2003,2), and that remains the 
> case if I put "?", ".", "NA" or "foo" in place of "TEST_HERE". A gap appears 
> only if I substitute "nan" or equivalent.
>
> There's a practical consequence: if I'm in the habit of writing "?" for 
> missing values in gnuplot input data, the doc would give me the idea that I 
> could toggle "gaps in lines" behavior by inserting or
> removing
>
> set datafile missing "?"
>
> With this set I should get no gaps, without it I should get gaps.
> But get no gaps regardless.

This proposed action is triggered by a recent support request for use of
gnuplot from the linux console command line (no X11 present).

Please contribute any comments or anecdotes about using gnuplot from the
linux console.  


Proposed action
===============

Version 5.2: Mark both terminals DEPRECATED
Version 5.3: Remove the configuration option --with-linux-vga
             Remove linux.trm and vgagl.trm from the current source

Background
==========

Both the "linux" and "vgagl" terminals rely on gnuplot being able to
write directly to the system graphics device (/dev/vga or /dev/svga or
/dev/console or various other synonyms).  This is accomplished by calling
into libraries libvga or libsvga (both are provided by the svgalib
source). The gnuplot executable must be installed suid root so that it
has sufficient privilege for this to work.  A corresponding linux
kernel module svgalib_helper must be available and loaded into the kernel.

This was reasonable 20 years ago; now, not so much.  

Nothing in a typical modern linux system requires or uses svgalib and
most distros do not include it by default.  It is not clear that it
is even possible to build it from existing source to work with current
linux kernels.  I have not been able to get it to work since gnuplot
version 4.2 ten years ago, although I have not really tried very hard. 

The gnuplot core routines supporting pm3d color had an alternative
code path controlled by #ifdef EXTENDED_COLOR_SPECS that affected all
terminals if the linux vgagl terminal was configured.  This code path
has not been maintained and no longer works in 5.3 so I removed it.
That means currently the vgagl terminal would not compile even if a
working svgalib were available. 


Replacement
===========

"set term dumb" is an option of course, but we can do better.

Various terminal interfaces can be run at the linux console level.
At least one of these (yaft) supports sixel graphics.
Therefore one way to run gnuplot from the linux console is to
run yaft (https://github.com/uobikiemukot/yaft) as your console
interface, run gnuplot from the yaft command line, and select
"set term sixel" for output.  I was able to do this successfully
on a laptop running linux kernel 4.17 and no special configuration
of either yaft or gnuplot. This seems an adequate replacement
for the original vgagl terminal, although neither of the current
gnuplot sixel output modes supports mousing.

On Sunday, July 29, 2018 10:37:49 PM PDT sfeam via gnuplot-beta wrote:
> On Saturday, 28 July 2018 22:35:00 Juhász Péter wrote:
> > Hello,
> > 
> > I wanted to create a graph where tics on the colorbox are dynamically
> > generated from data. I vaguely recalled that there was a feature for
> > that, and it seemed logical that if there is {x|y}[2]ticlabels() to
> > place tics on the main axes, there should be a cbticlabels to do the
> > same for the colorbox.
> > 
> > It seems that this is valid syntax, as the parser accepts it, and I've
> > found the code responsible for it in the source, but it appears to be
> > undocumented, and there are no demos for it either.
> > 
> > It also appears to be either half-finished or buggy. 
> 
> All true.  "cbticlabels" was added for completeness when the other
> ticlabels variants were introduced, but it was never fully thought
> out or documented. 
> 
> [snip]
> > See the attached data file and gnuplot script for a self-contained
> > example.
> > 
> > The fix, at least from the user's perspective, seems simple: make
> > cbticlabels use the same column that is used for the color data itself.
> > From the developer's perspective, I realize that this might be hard,
> > because we don't yet know which column to use at the time of parsing
> > the using spec, because an unknown number of `xxx variable` statements
> > may come later.
> > 
> > Or - `help pointsize variable` states that variable color is alwasy
> > taken from the last additional column. That we do know.
> 
> Yes.  I think that would be sufficient to cover all your test cases
> in 2D plots. Like this:
> 
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> diff --git a/src/datafile.c b/src/datafile.c
> index 8baf1d5..208ca3a 100644
> --- a/src/datafile.c
> +++ b/src/datafile.c
> @@ -2059,7 +2059,7 @@ df_readascii(double v[], int max)
>                             break;
>                         case CT_CBTICLABEL:
>                             axis = COLOR_AXIS;
> -                       axcol = 3;
> +                     axcol = df_no_use_specs - 1;
>                             break;
>                     }
>                     /* Trap special case of only a single 'using' column */
> @@ -5315,7 +5315,7 @@ df_readbinary(double v[], int max)
>                         break;
>                     case CT_CBTICLABEL:
>                         axis = COLOR_AXIS;
> -                   axcol = 2;
> +                 axcol = df_no_use_specs - 1;
>                         break;
>                 }
>                 if (a.type == STRING) {
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> 
> However that fix doesn't work so well for 3D plots, where there is always
> a z coordinate and that is usually (but not always) where the color comes
> from.  The code in datafile.c is shared by both 2D and 3D plots so it 
> would be tricky to make it work for both.
> 
> 	Ethan

I commited that change for 5.3 and added a documentation section noting
that it is EXPERIMENTAL and that the 3D heat map case does not work as it
probably should.    

	Ethan

On Saturday, 28 July 2018 22:35:00 Juhász Péter wrote:
> Hello,
> 
> I wanted to create a graph where tics on the colorbox are dynamically
> generated from data. I vaguely recalled that there was a feature for
> that, and it seemed logical that if there is {x|y}[2]ticlabels() to
> place tics on the main axes, there should be a cbticlabels to do the
> same for the colorbox.
> 
> It seems that this is valid syntax, as the parser accepts it, and I've
> found the code responsible for it in the source, but it appears to be
> undocumented, and there are no demos for it either.
> 
> It also appears to be either half-finished or buggy. 

All true.  "cbticlabels" was added for completeness when the other
ticlabels variants were introduced, but it was never fully thought
out or documented. 

[snip]
> See the attached data file and gnuplot script for a self-contained
> example.
> 
> The fix, at least from the user's perspective, seems simple: make
> cbticlabels use the same column that is used for the color data itself.
> From the developer's perspective, I realize that this might be hard,
> because we don't yet know which column to use at the time of parsing
> the using spec, because an unknown number of `xxx variable` statements
> may come later.
> 
> Or - `help pointsize variable` states that variable color is alwasy
> taken from the last additional column. That we do know.

Yes.  I think that would be sufficient to cover all your test cases
in 2D plots. Like this:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
diff --git a/src/datafile.c b/src/datafile.c
index 8baf1d5..208ca3a 100644
--- a/src/datafile.c
+++ b/src/datafile.c
@@ -2059,7 +2059,7 @@ df_readascii(double v[], int max)
                            break;
                        case CT_CBTICLABEL:
                            axis = COLOR_AXIS;
-                       axcol = 3;
+                     axcol = df_no_use_specs - 1;
                            break;
                    }
                    /* Trap special case of only a single 'using' column */
@@ -5315,7 +5315,7 @@ df_readbinary(double v[], int max)
                        break;
                    case CT_CBTICLABEL:
                        axis = COLOR_AXIS;
-                   axcol = 2;
+                 axcol = df_no_use_specs - 1;
                        break;
                }
                if (a.type == STRING) {
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

However that fix doesn't work so well for 3D plots, where there is always
a z coordinate and that is usually (but not always) where the color comes
from.  The code in datafile.c is shared by both 2D and 3D plots so it 
would be tricky to make it work for both.

	Ethan


> Best regards,
> Peter Juhasz

Hello,

I wanted to create a graph where tics on the colorbox are dynamically
generated from data. I vaguely recalled that there was a feature for
that, and it seemed logical that if there is {x|y}[2]ticlabels() to
place tics on the main axes, there should be a cbticlabels to do the
same for the colorbox.

It seems that this is valid syntax, as the parser accepts it, and I've
found the code responsible for it in the source, but it appears to be
undocumented, and there are no demos for it either.

It also appears to be either half-finished or buggy. A straightforward
invocation of `plot fn u 1:2:3:cbticlabels(4)` doesn't work, all tics
are placed at position 0 (so in effect there is only one tic label, the
last placed). I have to write `plot fn u 1:2:3:3:cbticlabels(4)`,
because the tic positions come from the 4th column specification. I see
in the source that this is hardcoded with a FIXME comment.

This behavior is problematic because it's confusing and goes against
expectations, and it produces incorrect results when there are multiple
variable specifications in the plot command (`ps var pt var lc pal`),
or with plot styles that reserve the fourth column for something else
already.
See the attached data file and gnuplot script for a self-contained
example.

The fix, at least from the user's perspective, seems simple: make
cbticlabels use the same column that is used for the color data itself.
>From the developer's perspective, I realize that this might be hard,
because we don't yet know which column to use at the time of parsing
the using spec, because an unknown number of `xxx variable` statements
may come later.

Or - `help pointsize variable` states that variable color is alwasy
taken from the last additional column. That we do know.

Best regards,
Peter Juhasz

sfeam <sf...@us...> writes:

> I think you are mis-interpreting the normal operation of PM3D as
> interpolation.  PM3D operates on quadrangles.  Each quadrangle has
> four corners.  Each corner has a color.   What color should be used for
> the quadrangle?   There are many choices and the command that selects one
> of the options is
>
>     set pm3d corners2color
> 		{ mean|geomean|harmean|rms|median|min|max|c1|c2|c3|c4 }
>
> The default option is "mean", and I guess that is what you are calling
> interpolation.  If you want each grid point in your original data to
> be the sole color source for a pm3d quadrangle ("pixel" if you are
> thinking of this as an image) then you want
>    set pm3d corners2color c1     (or c2 or c3 or c4)

Aha. Yes. That does sorta what I want. Except it shifts the image by 0.5
pixels in each direction. I understand why, but it's not ideal.


>> >> is there overhead for using pm3d in this case? Size? Speed? If there's
>> >> no overhead, can 'splot with image' simply map to 'splot with pm3d'?
>
> Many of the output devices can handle image data as a special case.
> The corresponding gnuplot driver can define a rectangular area and then
> pass only an array of color values to the device.  This is obviously a much
> smaller output stream than if the driver must output separate coordinates
> and color information for every pixel.  So in general the size of the
> output stream increases in the order
>
>    "with image" < "with pm3d" ~= "with image pixels"

OK. I forgot "with image pixels" existed. And just found an unrelated
(likely) bug: plotting the test data in this thread "with image" and
"with image pixels" produces slightly different images: the "with image"
version has each pixel centered not-quite at the correct locations. I
bet there's an off-by-one error in the "with image" code that makes the
coordinate mapping off slightly. I'll try to look at it when I get the
chance.


> I am thinking that we could use the command
>
>     set pm3d {explicit|implicit}
>
> as a model for opt-in contours.  As it is now, "set contours" causes
> all subsequent plots to use contours unless they cannot be contoured
> or they opt out with "nocontour".  This is the equivalent of the
> state produced by "set pm3d implicit".   It might work to add a command
> "set contours explicit" as we have for pm3d, meaning that subsequent
> plots are only contoured if they use an explicit plot style "contours".
>
> I don't think such a change would allow you to create any plot that you
> cannot already create, but perhaps making "set pm3d" and "set contours"
> more similar would be easier them both easier to understand.
> What do you think?

I think that extra bit of user control would definitely be welcome.

Thanks.
dima

On Saturday, 21 July 2018 16:49:32 sfeam via gnuplot-beta wrote:
> On Saturday, 21 July 2018 12:29:32 Dima Kogan wrote:

> > Yeah, this is an unfortunate consequence of the way we set up our
> > interface. Ideally, you'd specify the contour options per plot:
> > 
> >   splot A with lines contours, B with image contours, C with vectors
> >
> > Changing this retroactively is probably more trouble than it's worth.
> 
> Right.  The historical design for several of gnuplot's plotting modes
> was an all-or-nothing toggle.  "set hidden3d"  "set pm3d" "set contours".
> The code evolved to allow you to opt out of the global settings for
> individual plot components.  E.g.
> 
>    set hidden3d
>    splot FOO, BAZ with labels nohidden3d
> 
> guarantees that the labels in BAZ will not be obscured by the surface
> drawn for FOO.
> And for contours:
> 
>    set contour surface
>    splot  f(x,y),  g(x,y) nocontour, h(x,y) nosurface
> 
> produces a plot in which f() gets both surface and contours,
> g() gets surface only, and h() gets contours only.
> 
> pm3d evolved slightly differently. Instead of requiring a global
> setting and allowing plot components to opt out,  we now allow
> individual plot components to opt in by using the style "with pm3d".
> Yeah that's a bit inconsistent.  You might logically expect there to
> be a "nopm3d" option, but there isn't.

I am thinking that we could use the command

    set pm3d {explicit|implicit}

as a model for opt-in contours.  As it is now, "set contours" causes
all subsequent plots to use contours unless they cannot be contoured
or they opt out with "nocontour".  This is the equivalent of the 
state produced by "set pm3d implicit".   It might work to add a command
"set contours explicit" as we have for pm3d, meaning that subsequent
plots are only contoured if they use an explicit plot style "contours".

I don't think such a change would allow you to create any plot that you
cannot already create, but perhaps making "set pm3d" and "set contours"
more similar would be easier them both easier to understand.
What do you think?    

	Ethan

On Saturday, 21 July 2018 12:29:32 Dima Kogan wrote:
> Thanks Karl-Friedrich, Ethan for the suggestions.
> 
> These both sound like they work, but they also sound like workarounds
> instead of solutions. I have already found an ugly workaround that
> works: splot the same data twice: once 'with image' (for the colormap)
> and again 'with lines nosurface' (for the contours).
> 
> Comments inline.
> 
> 
> Ethan A Merritt <sf...@us...> writes:
> 
> > On Wednesday, July 18, 2018 12:37:17 PM PDT Dima Kogan wrote:
> >> Hi. I have a matrix data set, and I'd like to plot it using colors (with
> >> image), AND I'd like to overlay contours on top. This apparently does
> >> not work.
> >>
> >> It looks like 3d 'with image' plots use the data values for colors, but
> >> NOT for z. I.e. you get a colored image at z=0. Since it looks like
> >> contours read the z coordinate to do their thing, this doesn't work.
> >
> > I will try to explain why "with image" is not a plot style that can be
> > contoured.
> >
> > Consider a typical image file: PNG or GIF or JPEG or whatever.
> > It contains red/green/blue and maybe alpha values for each pixel.
> > But no z.  Where would you get a z value from?
> >
> > <snip>
> >
> > Now it is true that gnuplot's "with image" mode can be used for
> > gridded data where the color information is generated by
> > index lookup from gnuplot's own continuous color palette.
> > I guess that's what you are doing, but it is certainly not the general case.
> 
> In all my experience, I've used 'with rgbimage' for image files (.png,
> .jpg, ...) and 'with image' for gridded data. And 'help image' describes
> the gridded data use case first, and at least this suggests that this
> isn't some sort of side-feature.

It's not a side-feature.  It is a method for plotting color-indexed
images, e.g. GIF or non-truecolor PNG.   But indexed color is not the
same thing as "z values".  There are no z values in a GIF file.


> >> pm3d is intended for irregularly-sampled data (as I understand it)
> >
> > Not really. pm3d is a coloring algorithm that can be applied to
> > anything constructured from quadrangles. Typically this is a regularly
> > gridded surface. There are also options for interpolating an
> > irregularly sampled data set to generate a regular grid that can then
> > be colored (see "help dgrid3d"). But it sounds like your case doesn't
> > need any extra step to regularize the grid.
> 
> Without extra options pm3d was re-interpolating my data in some way, so
> its results weren't the same as 'with image'. And it's doing something
> different internally, and ends up producing bigger (and presumably less
> efficient) output.

I think you are mis-interpreting the normal operation of PM3D as
interpolation.  PM3D operates on quadrangles.  Each quadrangle has
four corners.  Each corner has a color.   What color should be used for
the quadrangle?   There are many choices and the command that selects one
of the options is 

    set pm3d corners2color 
		{ mean|geomean|harmean|rms|median|min|max|c1|c2|c3|c4 }

The default option is "mean", and I guess that is what you are calling
interpolation.  If you want each grid point in your original data to
be the sole color source for a pm3d quadrangle ("pixel" if you are
thinking of this as an image) then you want
   set pm3d corners2color c1     (or c2 or c3 or c4)

  
> >> is there overhead for using pm3d in this case? Size? Speed? If there's
> >> no overhead, can 'splot with image' simply map to 'splot with pm3d'?

Many of the output devices can handle image data as a special case.
The corresponding gnuplot driver can define a rectangular area and then
pass only an array of color values to the device.  This is obviously a much
smaller output stream than if the driver must output separate coordinates
and color information for every pixel.  So in general the size of the
output stream increases in the order

   "with image" < "with pm3d" ~= "with image pixels"

> >> If nothing else, can we barf instead of silently not rendering the
> >> contours in the 'with image' case?
> >
> > The more common complaint is that too many error messages
> > are printed for things that are not fatal errors.
> > Consider a hypothetical plot that includes some contours, some image data,
> > and some other stuff:
> >
> >     set contour
> >     splot A with lines, B with image, C with vectors
> >
> > The result is a plot with contours for A and no contours for B or C.
> > What would be the benefit of spewing errors or failing just because
> > B and C are plotted using styles that cannot be contoured?
> 
> Yeah, this is an unfortunate consequence of the way we set up our
> interface. Ideally, you'd specify the contour options per plot:
> 
>   splot A with lines contours, B with image contours, C with vectors
>
> Changing this retroactively is probably more trouble than it's worth.

Right.  The historical design for several of gnuplot's plotting modes
was an all-or-nothing toggle.  "set hidden3d"  "set pm3d" "set contours".
The code evolved to allow you to opt out of the global settings for
individual plot components.  E.g.

   set hidden3d
   splot FOO, BAZ with labels nohidden3d

guarantees that the labels in BAZ will not be obscured by the surface
drawn for FOO.
And for contours:

   set contour surface
   splot  f(x,y),  g(x,y) nocontour, h(x,y) nosurface

produces a plot in which f() gets both surface and contours,
g() gets surface only, and h() gets contours only.

pm3d evolved slightly differently. Instead of requiring a global
setting and allowing plot components to opt out,  we now allow
individual plot components to opt in by using the style "with pm3d".
Yeah that's a bit inconsistent.  You might logically expect there to
be a "nopm3d" option, but there isn't.

   
> OK. Let me look at the code and see if I can make 'with image' do what I
> think it should do. Don't know when I'll get to it, but that will make
> this into a more concrete discussion.

It _cannot_ do what you want, because in general image data does not come
with z values.

	Ethan

> 
> dima

Thanks Karl-Friedrich, Ethan for the suggestions.

These both sound like they work, but they also sound like workarounds
instead of solutions. I have already found an ugly workaround that
works: splot the same data twice: once 'with image' (for the colormap)
and again 'with lines nosurface' (for the contours).

Comments inline.


Ethan A Merritt <sf...@us...> writes:

> On Wednesday, July 18, 2018 12:37:17 PM PDT Dima Kogan wrote:
>> Hi. I have a matrix data set, and I'd like to plot it using colors (with
>> image), AND I'd like to overlay contours on top. This apparently does
>> not work.
>>
>> It looks like 3d 'with image' plots use the data values for colors, but
>> NOT for z. I.e. you get a colored image at z=0. Since it looks like
>> contours read the z coordinate to do their thing, this doesn't work.
>
> I will try to explain why "with image" is not a plot style that can be
> contoured.
>
> Consider a typical image file: PNG or GIF or JPEG or whatever.
> It contains red/green/blue and maybe alpha values for each pixel.
> But no z.  Where would you get a z value from?
>
> <snip>
>
> Now it is true that gnuplot's "with image" mode can be used for
> gridded data where the color information is generated by
> index lookup from gnuplot's own continuous color palette.
> I guess that's what you are doing, but it is certainly not the general case.

In all my experience, I've used 'with rgbimage' for image files (.png,
.jpg, ...) and 'with image' for gridded data. And 'help image' describes
the gridded data use case first, and at least this suggests that this
isn't some sort of side-feature.


>> If I plot 'with lines', I get contours. I can also plot 'with pm3d',
>> which produces a color AND a z-coordinate, so contours work then.
>>
>> Questions:
>>
>> It makes no sense to me that 'with image' 3d plots are stuck at z=0.
>> Does ANYBODY want that?
>
> They are not stuck at z=0.  See for example plots 6-8 of the image2 demo:
>
>       http://gnuplot.sourceforge.net/demo_5.2/image2.html

All right. If I specify something like "rotate", will I get contours? If
not, then I claim that this is surprising to the user, and is thus a
bug. My fundamental complaint is this:

  splot "something" with pm3d

gives me contours. 'with lines' and 'with points' give me contours.
'with image' does NOT give me contours. Silently. To a USER who doesn't
know or care about the gnuplot internals this is a bug.


>> Can we apply the values to color AND z in this
>> case? Is this hard, or is there some specific reason we're doing it the
>> way we're doing it?
>>
>> pm3d is intended for irregularly-sampled data (as I understand it)
>
> Not really. pm3d is a coloring algorithm that can be applied to
> anything constructured from quadrangles. Typically this is a regularly
> gridded surface. There are also options for interpolating an
> irregularly sampled data set to generate a regular grid that can then
> be colored (see "help dgrid3d"). But it sounds like your case doesn't
> need any extra step to regularize the grid.

Without extra options pm3d was re-interpolating my data in some way, so
its results weren't the same as 'with image'. And it's doing something
different internally, and ends up producing bigger (and presumably less
efficient) output.


>> is there overhead for using pm3d in this case? Size? Speed? If there's
>> no overhead, can 'splot with image' simply map to 'splot with pm3d'?
>
> In gnuplot version 5 you can tell pm3d to use a pre-calculated
> RGB color for each grid point.  This RGB value must be in the form of
> a 24-bit packed RGB integer.   I don't know exactly what  the nature of
> the values in your matrix so I cannot provide exact instructions,
> but in essence the command sequence would be
>
>   set pm3d border lc rgb variable
>   set style fill solid 1.0
>   splot $DATA using 1:2:3:(rgbfunc($3)) with pm3d
>
> It is up to you to define rgbfunc() appropriately.
> Here is an example using a regularly sampled grid and a constant
> color value:
>
>   set xrange [-2:2]
>   set yrange [-2:2]
>   set sample 20
>   set isosample 20
>
>   splot '++' using 1:2:(cos(y)*sin(x)):(int(0xdd77aa)) with pm3d
>
> But your special case may not require any of that.
> If the "z" values on a regular grid are to be used also as a  palette
> index then indeed the default pm3d settings may be sufficient.

Thanks for the example. I'll keep this usage in mind for the future, but
it's not applicable to my immediate use case: I have 1-dimensional
values, so the default gnuplot colormapping is what I want.


>> If nothing else, can we barf instead of silently not rendering the
>> contours in the 'with image' case?
>
> The more common complaint is that too many error messages
> are printed for things that are not fatal errors.
> Consider a hypothetical plot that includes some contours, some image data,
> and some other stuff:
>
>     set contour
>     splot A with lines, B with image, C with vectors
>
> The result is a plot with contours for A and no contours for B or C.
> What would be the benefit of spewing errors or failing just because
> B and C are plotted using styles that cannot be contoured?

Yeah, this is an unfortunate consequence of the way we set up our
interface. Ideally, you'd specify the contour options per plot:

  splot A with lines contours, B with image contours, C with vectors

Changing this retroactively is probably more trouble than it's worth.

OK. Let me look at the code and see if I can make 'with image' do what I
think it should do. Don't know when I'll get to it, but that will make
this into a more concrete discussion.

dima

On Wednesday, July 18, 2018 12:37:17 PM PDT Dima Kogan wrote:
> Hi. I have a matrix data set, and I'd like to plot it using colors (with
> image), AND I'd like to overlay contours on top. This apparently does
> not work.
> 
> It looks like 3d 'with image' plots use the data values for colors, but
> NOT for z. I.e. you get a colored image at z=0. Since it looks like
> contours read the z coordinate to do their thing, this doesn't work.

I will try to explain why "with image" is not a plot style that can be
contoured.

Consider a typical image file: PNG or GIF or JPEG or whatever.
It contains red/green/blue and maybe alpha values for each pixel.
But no z.  Where would you get a z value from?

It is true that PNG or GIF can also encode indexed color.
But the sequentially indexed color values do not necessarily correspond
to a continuous palette like the ones that gnuplot can generate.
So the matrix of index values is not generally something that can
be meaningfully contoured. 

Now it is true that gnuplot's "with image" mode can be used for 
gridded data where the color information is generated by
index lookup from gnuplot's own continuous color palette.
I guess that's what you are doing, but it is certainly not the general case.

> If I plot 'with lines', I get contours. I can also plot 'with pm3d',
> which produces a color AND a z-coordinate, so contours work then.
> 
> Questions:
> 
> It makes no sense to me that 'with image' 3d plots are stuck at z=0.
> Does ANYBODY want that? 

They are not stuck at z=0.  See for example plots 6-8 of the image2 demo:

      http://gnuplot.sourceforge.net/demo_5.2/image2.html
    
> Can we apply the values to color AND z in this
> case? Is this hard, or is there some specific reason we're doing it the
> way we're doing it?
> 
> pm3d is intended for irregularly-sampled data (as I understand it)

Not really. pm3d is a coloring algorithm that can be applied to anything
constructured from quadrangles.  Typically this is a regularly gridded surface.
There are also options for interpolating an irregularly sampled data set to
generate a regular grid that can then be colored (see "help dgrid3d"). 
But it sounds like your case doesn't need any extra step to regularize the grid.

> is there overhead for using pm3d in this case? Size? Speed? If there's
> no overhead, can 'splot with image' simply map to 'splot with pm3d'?

In gnuplot version 5 you can tell pm3d to use a pre-calculated
RGB color for each grid point.  This RGB value must be in the form of
a 24-bit packed RGB integer.   I don't know exactly what  the nature of
the values in your matrix so I cannot provide exact instructions,
but in essence the command sequence would be

  set pm3d border lc rgb variable
  set style fill solid 1.0
  splot $DATA using 1:2:3:(rgbfunc($3)) with pm3d

It is up to you to define rgbfunc() appropriately.
Here is an example using a regularly sampled grid and a constant
color value:

  set xrange [-2:2]
  set yrange [-2:2]
  set sample 20
  set isosample 20

  splot '++' using 1:2:(cos(y)*sin(x)):(int(0xdd77aa)) with pm3d

But your special case may not require any of that.
If the "z" values on a regular grid are to be used also as a  palette
index then indeed the default pm3d settings may be sufficient.

> If nothing else, can we barf instead of silently not rendering the
> contours in the 'with image' case?

The more common complaint is that too many error messages
are printed for things that are not fatal errors.
Consider a hypothetical plot that includes some contours, some image data,
and some other stuff:

    set contour
    splot A with lines, B with image, C with vectors

The result is a plot with contours for A and no contours for B or C.  
What would be the benefit of spewing errors or failing just because
B and C are plotted using styles that cannot be contoured?

	Ethan


> Thanks!
> 
>

Dima Kogan <gn...@di...> writes:

> It makes no sense to me that 'with image' 3d plots are stuck at z=0.
> Does ANYBODY want that? Can we apply the values to color AND z in this
> case? Is this hard, or is there some specific reason we're doing it the
> way we're doing it?
>
> pm3d is intended for irregularly-sampled data (as I understand it), so
> is there overhead for using pm3d in this case? Size? Speed? If there's
> no overhead, can 'splot with image' simply map to 'splot with pm3d'?

I just answered some of this. Plotting to a pdf, the pmd3d files are 70%
larger. And it looks like pm3d re-interpolated the original data, which
I don't want, and which shouldn't be needed, since I already have
regularly-spaced matrix data.

So the question that remains is the one about making 'with image'
produce colors AND z