Open
Description
The below example fails after working for 5 minutes with its current radius = 1.5 degrees, but works with a smaller radius (1.5 arcsec). Not sure where the breaking point is. I don't consider 1.5 degrees to be a very large radius, and is motivated by my science use case. I don't know how big the table is that query_ssa is accessing, but it also seems like 5 min is a long time for this search. I think the spitzer_irsenh catalog is only a few tens of thousands of sources so, if the filtering happens first, the 5 min is extra suspicious. I ran the code twice to make this example, and got two different tracebacks with similar but not exactly the same messages, so both are below.
import astropy.coordinates as coord
import astropy.units as u
from astroquery.ipac.irsa import Irsa
NEP_RA = 269.31 #deg
NEP_dec = 66.47 #deg
NEP_skycoord = coord.SkyCoord(ra=NEP_RA, dec=NEP_dec,
unit=(u.deg, u.deg),
frame='icrs')
IRS_spectra_table = Irsa.query_ssa(pos=NEP_skycoord,
radius = 1.5*u.deg, #within 1.5 degrees of the NEP
collection='spitzer_irsenh') #does it have a spectrum available
first traceback:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:260](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=259), in DALQuery.execute_votable(self, post)
259 try:
--> 260 return votableparse(self.execute_stream(post=post).read)
261 except Exception as e:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/astropy/io/votable/table.py:164](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/astropy/io/votable/table.py#line=163), in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)
161 with iterparser.get_xml_iterator(
162 source, _debug_python_based_parser=_debug_python_based_parser
163 ) as iterator:
--> 164 return tree.VOTableFile(config=config, pos=(1, 1)).parse(iterator, config)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/astropy/io/votable/tree.py:4249](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/astropy/io/votable/tree.py#line=4248), in VOTableFile.parse(self, iterator, config)
4247 config["_current_table_number"] = 0
-> 4249 for start, tag, data, pos in iterator:
4250 if start:
ValueError: 1:49: syntax error
During handling of the above exception, another exception occurred:
DALServiceError Traceback (most recent call last)
Cell In[5], line 5
1 NEP_skycoord = coord.SkyCoord(ra=NEP_RA, dec=NEP_dec,
2 unit=(u.deg, u.deg),
3 frame='icrs')
4 #want to filter on a certain radius and if it has an actual spectrum in the IRS enhanced archive
----> 5 IRS_spectra_table = Irsa.query_ssa(pos=NEP_skycoord,
6 radius = 1.5*u.deg, #within 1.5 degrees of the NEP
7 collection='spitzer_irsenh') #does it have a spectrum available
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/astroquery/ipac/irsa/core.py:164](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/astroquery/ipac/irsa/core.py#line=163), in IrsaClass.query_ssa(self, pos, radius, band, time, collection)
161 else:
162 diameter = 2 * radius
--> 164 results = self.ssa.search(pos=pos, diameter=diameter, band=band, time=time,
165 format='all', collection=collection)
166 return results.to_table()
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py:213](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py#line=212), in SSAService.search(self, pos, diameter, band, time, format, **keywords)
162 def search(
163 self, pos=None, *, diameter=None, band=None, time=None, format=None,
164 **keywords):
165 """
166 submit a SSA query to this service with the given constraints.
167
(...) 210 pyvo.dal.DALQueryError
211 """
212 return self.create_query(
--> 213 pos=pos, diameter=diameter, band=band, time=time, format=format, **keywords).execute()
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py:576](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py#line=575), in SSAQuery.execute(self)
562 def execute(self):
563 """
564 submit the query and return the results as a SSAResults instance
565
(...) 574 for errors parsing the VOTable response
575 """
--> 576 return SSAResults(self.execute_votable(), url=self.queryurl, session=self._session)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:262](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=261), in DALQuery.execute_votable(self, post)
260 return votableparse(self.execute_stream(post=post).read)
261 except Exception as e:
--> 262 self.raise_if_error()
263 raise DALFormatError(e, self.queryurl)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:271](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=270), in DALQuery.raise_if_error(self)
269 if self._ex:
270 e = self._ex
--> 271 raise DALServiceError.from_except(e, self.queryurl)
DALServiceError: 502 Server Error: Proxy Error for url: https://irsa.ipac.caltech.edu/SSA?POS=269.31%2C66.47&SIZE=3.0&FORMAT=all&REQUEST=queryData&COLLECTION=spitzer_irsenh
second traceback:
---------------------------------------------------------------------------
RemoteDisconnected Traceback (most recent call last)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py:787](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py#line=786), in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)
786 # Make the request on the HTTPConnection object
--> 787 response = self._make_request(
788 conn,
789 method,
790 url,
791 timeout=timeout_obj,
792 body=body,
793 headers=headers,
794 chunked=chunked,
795 retries=retries,
796 response_conn=response_conn,
797 preload_content=preload_content,
798 decode_content=decode_content,
799 **response_kw,
800 )
802 # Everything went great!
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py:534](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py#line=533), in HTTPConnectionPool._make_request(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)
533 try:
--> 534 response = conn.getresponse()
535 except (BaseSSLError, OSError) as e:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connection.py:516](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connection.py#line=515), in HTTPConnection.getresponse(self)
515 # Get the response from http.client.HTTPConnection
--> 516 httplib_response = super().getresponse()
518 try:
File [/opt/conda/envs/notebook/lib/python3.12/http/client.py:1411](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/http/client.py#line=1410), in HTTPConnection.getresponse(self)
1410 try:
-> 1411 response.begin()
1412 except ConnectionError:
File [/opt/conda/envs/notebook/lib/python3.12/http/client.py:324](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/http/client.py#line=323), in HTTPResponse.begin(self)
323 while True:
--> 324 version, status, reason = self._read_status()
325 if status != CONTINUE:
File [/opt/conda/envs/notebook/lib/python3.12/http/client.py:293](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/http/client.py#line=292), in HTTPResponse._read_status(self)
290 if not line:
291 # Presumably, the server closed the connection before
292 # sending a valid response.
--> 293 raise RemoteDisconnected("Remote end closed connection without"
294 " response")
295 try:
RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
ProtocolError Traceback (most recent call last)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/adapters.py:667](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/adapters.py#line=666), in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
666 try:
--> 667 resp = conn.urlopen(
668 method=request.method,
669 url=url,
670 body=request.body,
671 headers=request.headers,
672 redirect=False,
673 assert_same_host=False,
674 preload_content=False,
675 decode_content=False,
676 retries=self.max_retries,
677 timeout=timeout,
678 chunked=chunked,
679 )
681 except (ProtocolError, OSError) as err:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py:841](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py#line=840), in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)
839 new_e = ProtocolError("Connection aborted.", new_e)
--> 841 retries = retries.increment(
842 method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
843 )
844 retries.sleep()
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/util/retry.py:474](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/util/retry.py#line=473), in Retry.increment(self, method, url, response, error, _pool, _stacktrace)
473 if read is False or method is None or not self._is_method_retryable(method):
--> 474 raise reraise(type(error), error, _stacktrace)
475 elif read is not None:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/util/util.py:38](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/util/util.py#line=37), in reraise(tp, value, tb)
37 if value.__traceback__ is not tb:
---> 38 raise value.with_traceback(tb)
39 raise value
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py:787](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py#line=786), in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)
786 # Make the request on the HTTPConnection object
--> 787 response = self._make_request(
788 conn,
789 method,
790 url,
791 timeout=timeout_obj,
792 body=body,
793 headers=headers,
794 chunked=chunked,
795 retries=retries,
796 response_conn=response_conn,
797 preload_content=preload_content,
798 decode_content=decode_content,
799 **response_kw,
800 )
802 # Everything went great!
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py:534](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connectionpool.py#line=533), in HTTPConnectionPool._make_request(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)
533 try:
--> 534 response = conn.getresponse()
535 except (BaseSSLError, OSError) as e:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connection.py:516](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/urllib3/connection.py#line=515), in HTTPConnection.getresponse(self)
515 # Get the response from http.client.HTTPConnection
--> 516 httplib_response = super().getresponse()
518 try:
File [/opt/conda/envs/notebook/lib/python3.12/http/client.py:1411](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/http/client.py#line=1410), in HTTPConnection.getresponse(self)
1410 try:
-> 1411 response.begin()
1412 except ConnectionError:
File [/opt/conda/envs/notebook/lib/python3.12/http/client.py:324](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/http/client.py#line=323), in HTTPResponse.begin(self)
323 while True:
--> 324 version, status, reason = self._read_status()
325 if status != CONTINUE:
File [/opt/conda/envs/notebook/lib/python3.12/http/client.py:293](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/http/client.py#line=292), in HTTPResponse._read_status(self)
290 if not line:
291 # Presumably, the server closed the connection before
292 # sending a valid response.
--> 293 raise RemoteDisconnected("Remote end closed connection without"
294 " response")
295 try:
ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
ConnectionError Traceback (most recent call last)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:260](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=259), in DALQuery.execute_votable(self, post)
259 try:
--> 260 return votableparse(self.execute_stream(post=post).read)
261 except Exception as e:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/utils/decorators.py:9](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/utils/decorators.py#line=8), in stream_decode_content.<locals>.wrapper(*args, **kwargs)
7 @wraps(func)
8 def wrapper(*args, **kwargs):
----> 9 raw = func(*args, **kwargs)
10 raw.read = partial(raw.read, decode_content=True)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:209](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=208), in DALQuery.execute_stream(self, post)
203 """
204 Submit the query and return the raw response as a file stream.
205
206 No exceptions are raised here because non-2xx responses might still
207 contain payload. They can be raised later by calling ``raise_if_error``
208 """
--> 209 response = self.submit(post=post)
211 try:
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:230](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=229), in DALQuery.submit(self, post)
229 else:
--> 230 response = self._session.get(url, params=params, stream=True,
231 allow_redirects=True)
232 return response
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/sessions.py:602](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/sessions.py#line=601), in Session.get(self, url, **kwargs)
601 kwargs.setdefault("allow_redirects", True)
--> 602 return self.request("GET", url, **kwargs)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/sessions.py:589](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/sessions.py#line=588), in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
588 send_kwargs.update(settings)
--> 589 resp = self.send(prep, **send_kwargs)
591 return resp
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/sessions.py:703](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/sessions.py#line=702), in Session.send(self, request, **kwargs)
702 # Send the request
--> 703 r = adapter.send(request, **kwargs)
705 # Total elapsed time of the request (approximately)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/adapters.py:682](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/requests/adapters.py#line=681), in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
681 except (ProtocolError, OSError) as err:
--> 682 raise ConnectionError(err, request=request)
684 except MaxRetryError as e:
ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
DALFormatError Traceback (most recent call last)
Cell In[2], line 12
7 NEP_skycoord = coord.SkyCoord(ra=NEP_RA, dec=NEP_dec,
8 unit=(u.deg, u.deg),
9 frame='icrs')
10 #want to filter on a certain radius and
11 # if it has an actual spectrum in the IRS enhanced archive
---> 12 IRS_spectra_table = Irsa.query_ssa(pos=NEP_skycoord,
13 radius = 1.5*u.deg, #within 1.5 degrees of the NEP
14 collection='spitzer_irsenh') #does it have a spectrum available
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/astroquery/ipac/irsa/core.py:164](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/astroquery/ipac/irsa/core.py#line=163), in IrsaClass.query_ssa(self, pos, radius, band, time, collection)
161 else:
162 diameter = 2 * radius
--> 164 results = self.ssa.search(pos=pos, diameter=diameter, band=band, time=time,
165 format='all', collection=collection)
166 return results.to_table()
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py:213](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py#line=212), in SSAService.search(self, pos, diameter, band, time, format, **keywords)
162 def search(
163 self, pos=None, *, diameter=None, band=None, time=None, format=None,
164 **keywords):
165 """
166 submit a SSA query to this service with the given constraints.
167
(...) 210 pyvo.dal.DALQueryError
211 """
212 return self.create_query(
--> 213 pos=pos, diameter=diameter, band=band, time=time, format=format, **keywords).execute()
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py:576](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/ssa.py#line=575), in SSAQuery.execute(self)
562 def execute(self):
563 """
564 submit the query and return the results as a SSAResults instance
565
(...) 574 for errors parsing the VOTable response
575 """
--> 576 return SSAResults(self.execute_votable(), url=self.queryurl, session=self._session)
File [/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py:263](https://science-console.fornax.smce.nasa.gov/opt/conda/envs/notebook/lib/python3.12/site-packages/pyvo/dal/query.py#line=262), in DALQuery.execute_votable(self, post)
261 except Exception as e:
262 self.raise_if_error()
--> 263 raise DALFormatError(e, self.queryurl)
DALFormatError: ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))